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ABSTRACT 


A  test  of  the  hypothesis  that  the  second  parameters 
("success"  probabilities)  of  a  number  of  paired  binomial 
distributions  are  pairwise  equal  is  derived  under  weak 
assumptions o   Computer  codes  necessary  to  implement  the 
procedure  are  given  and  a  case  study  is  used  to  demonstrate 
the  procedure 0   Some  other  procedures  for  testing  the  same 
hypothesis  under  stronger  assumptions  are  discussed  and 
compared  with  the  given  procedure .   A  rapid  approximate 
procedure  is  also  givenc 
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CHAPTER  I 

INTRODUCTION 

In  certain  experimental  situations  one  is  interested 
in  the  hypothesis  of  pairwise  equality  of  means  in  (say) 
n  pairs  of  binomial  distributions,  but  has  no  interest  in 
what  that  mean  (for  a  given  pair)  is.   Specifically,  if  we 
denote  the  "success  probability"  parameter  of  the  first 
member  of  the  ith  pair  by  d.  and  the  same  parameter  of  the 
second  member  by  f  ,  then  it  is  desired  to  test  the 
hypothesis 

HQ:  di  =  f1   i,  i  =  1,2,  ,.o,  n 

against  various  alternate  hypotheses  which  are  left  un- 
specified at  this  point.   Note  that  the  relationships  (if 

any)  between  d.  and  d,  or  between  f  and  f  for  i  4   J »  are 
i      j  i      j 

of  no  concern,  nor  are  the  numerical  values  of  d.  and  f  . 

i      i 

A  practical  example  of  the  need  for  this  type  hypo- 
thesis test  arises  in  the  examination  of  a  detection  system, 
A  basic  requirement  of  a  detection  system  is  that  it  should 
perform  better  than  a  "random"  system,  which  is  defined  as 
one  in  which  d,  the  probability  of  detecting  a  target 
(given  that  one  is  actually  present)  is  equal  to  f,  the 
probability  of  giving  a  false  alarm.   A  system  may  be  con- 
sidered better  than  random  if  d  > f . 


Suppose  we  have  a  system  which  may  respond  differently 
under  different  operators 0      If  d.  and  f.  are  the  detection 
and  false  alarm  probabilities 9   respectively,  for  the  itlrl 
operators  we  are  naturally  interested  in  testing 

H  :  d.  =  f    i;  i  =  1,28  „ e „ ,  n  against,  say 
Qi     l 

Hl:  di  >  fi   1»  1  ~  ls2?  9°°  »  n- 

In  some  situations,  one  might  require  a  two  sided  test  or 
perhaps  a  less  stringent  alternate  hypothesis. 

In  any  case9  the  matter  of  interest  is  whether  the 
system  performs  better  than  "random"  for  each  operator, 
without  regard  to  possible  differences  between  operators. 
For  the  remainder  of  this  paper,  the  detection  system 
example  will  serve  as  a  "prototype"  case.   It  is  hoped  that 
this  will  result  in  increased  intuitive  appeal  and  clarity 
of  presentation 

A  typical  experiment  conducted  to  facilitate  per- 
formance of  the  desired  hypothesis  test  involves  n  operators, 

th 
the  i   of  whom  makes  r.  detection  attempts  with  targets 

actually  present  and  s.  attempts  with  no  targets  present. 

The  data  then  consists  of  n  pairs  of  observations  (J  f3c  ) 

where  i   is  a  realization  of  the  random  variable  J  ,  the 
Ji  i» 

th 
number  of  true  detections  made  by  the  i   operator  from 

among  the  r  attempts  with  targets  present,  and  k  is  a 
i  i 

realization  of  the  random  variable  k    ,  the  number  of  false 
alarms  registered  from  among  the  s  attempts  made  without 
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th 
targets  present.   We  assume  the  i   operator  makes  true 

detections  with  constant  probability  d.  ,  and  generates 

false  alarms  with  constant  probability  f. .   Thus  J  is 

binomlally  distributed  with  "success"  probability  parameter 

d.  and  number  of  trials  parameter  r.  (hereafter  written 

J1  i   b(r  ,d  )).   Similarly,  we  have  JL    =   b(s  ,f  ). 

We  will  assume  that  the  experimental  design  and 

general  conditions  are  such  that  the  following  assumptions 

are  satisfied: 

1.  r  =s  =m  Vi  (made  only  for  mathematical  simplicity) 

2.  J  i  b(m,di),  K±   I  b(m,fi)  Vi;  1  =  1,2,  .„„,  n 
3o  J.  and  K.  are  independent  Vi;   1  =  1,2,  . «, .  ,  n 

k,  J  and  J .  are  independent  for  i  ^  j 

5.   K  and  K  are  independent  for  i  ^  j 
<■■      J 

SOME  EARLIER  APPROACHES 

Before  proceeding  with  our  development  of  the  desired 
hypothesis  test,  we  briefly  discuss  two  procedures  which 
have  been  used  in  the  paste   Since  both  of  these  procedures 
require  assumptions  stronger  than  those  listed  above,  they 
are  of  restricted  applicability,, 

The  Equal  Performance  Assumptione   If  the  additional  assump- 
tion that  d  =  d  V^  and  *\  =  f  V**  then  the  experimental 

data  may  be  summed  into  a  single  pair  of  observations 

n  n 

y   =     £  j.  and  k'  =  2  k.  where  y    and  k*  are,  respectively, 

i=l  x  i=l 

realizations  of  J'  and  K* ,  where  J*  !  b(nm,d)  and 

K*  4  b(nm,f). 
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The  natural  hypotheses  under  test  are  then 


HQs  d  ~  f  against  E    s  d>f, 


which  may  readily  be  tested  by  the  exact  method  discussed 

1  2 

below  or  by  an  appropriate  asymtotic  test  . 

A  hypothetical  example  serves  to  illustrate  the 

dangers  of  improper  application  of  the  foregoing  procedure. 

Suppose  that  n/2  of  the  observed  pairs  were  (m,0),  while 

the  other  n/2   were  (0,m)o   Then  j'  =  nm/2  and  k1  =  nm/2 

and  the  hypothesis  that  d  =  f  would  be  accepted,   ia/hat  has 

probably  happened ,  however  is  that  d.  >  f  for  about  half 

i   1 

the  operators  while  d,  <  f .  for  the  remainder.   Summing 

i   1 

over  ail  individuals  has  obscured  information  to  this 
effect  contained  in  the  datac 

In  an  extreme  case9  such  as  the  above,  the  experi- 
menter should  recognize  that  the  equal  performance  assump- 
tion does  not  hold0   However  suppose  the  case  were  less 
extreme 9    say  that  given  in  Table  1  (assume  m>6). 

TABLE  1 
Results  of  a  Hypothetical  Experiment 
Number  of  observations 

n/4-  (m  -  2,  4) 

nA  (m  -  6,  2) 

nA  (5,  m  -  3) 

n/Ur  (1,  m  -  5) 


Value  of  observation  (j.,k  ) 
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Here,  testing  under  the  assumption  of  equal  performance 
yields  y    =  (n  -  1  )m/2  and  k'  =  (n  -  1  )m/2 ,  and  again,  one 
would  accept  the  hypothesis  that  d  =  f.   rfith  this  result, 
the  experimenter  would  not  be  likely  to  suspect  the  equal 
performance  assumption,  yet  it  is  not  clear  that  it  is 
satisfied. 

One  might  attempt  to  "validate"  the  assumption  of  equal 
performance  by  conducting  a  homogeneity-'9   test  on  the  ob- 
servation vectors  (J^*J2  °,e»  ^n^  and  (ki»k2  •••'  k  ^  or  by 
goodness  of  fit  tests-5  to  some  binomial  distributions. 
However,  the  homogeneity  test  for  binomial  data  is  a  strictly 
intuitive  procedure,  and  thus  not  entirely  desireable;  also 
the  selection  of  which  binomial  distribution  to  test  the 
data  against  poses  a  considerable  problem,,   Further,  in  both 
cases,  the  size  of  the  overall  procedure  is  difficult  to 
determine.   *[e  therefore  conclude  this  discussion  by  noting 
that  testing  for  homogeneity  among  binomial  samples  is  an 
area  in  which  additional  work  appears  to  be  needed. 
The  Analysis  of  Variance  Approach.   Bartlett   has  suggested 
and  Curtiss'  has  shown  that  the  arcsine  transformation 
Y  =  arcsin  yX/5  where  X  =  b(m,p),  imparts  to  Y  an  approximate 
normal  distribution  with  mean  arcsin  Vp"  an<^  variance  l/4m. 
(Denoted  Y  ^  N(arcsin  «/p~  ,1/^)  °  )      Using  this  technique  in  ■ 
the  prototype  case  we  have  J?  =  arcsin  J  J   /m  and  K.'  =  arcsin 
VK. /m  distributed  approximately  normal  with  means  arcsin 
7  d.  and  arcsin  ,/f  respectively,  and  the  same  variance  l/4m. 
If  we  now  think  of  having  a  target  present  as  treatment  D 
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and  having  no  target  present  as  treatment  F*  the  J*  repre- 
sents a  "response"  under  treatment  D,  while  K*  represents  a 
response  under  treatment  F0   We  can  now  model  these  responses, 
say  J?  as  J!  =  *7   +  <p .  +  Cft/  where 

r^D     -   mean  {transformed)  response  under  treatment  D  » 

&■    =  differential  (transformed)  response  under  treat- 

*"h 
ment  D  for  the  i   operator 


Ot 


=  (transformed)  sampling  error  for  the  observation 

th 
on  the  i   operator  under  treatment  D. 


K*  could  be  modeled  analogously  as  ^,4-  ^-  ■+  €k;    and  we  can 
assume  that : 

2 
L^.t-  is  asymtotically  N(Oscrd  )  y  =   k,D;  i  =  l,2,...,n 

20  €w  is  asymtotically  N(0y<r2)  -v  =  k,D;  i  =  l,2,...,n. 

The  random  factor  models  two  way  classification  Analysis  of 
Variance  may  then  be  used  to  test 

H«  s  d  =  f  \/%     against  the  somewhat  restricted  and 

0  i    i 

unnatural  alternate  hypothesis 

EL  s  &,  4   f   for  some  !„ 

1  i    i 

A  further  problem  with  this  approach  is  that  it  requires 

m  "large"  and  d,  and  f  not  "extreme" ,  say  0o2<d.  <0«8  and 

i  I  r 

0o2<f„  <0o8   in  order  for  the  arcsine  transformation  to 

i 

yield  reasonably  normally  distributed  random  variables. 
Since  we  cannot  ( indeed ,  may  not  want  to)  assure  that  these 
additional  requirements  are  met,  this  approach  is  of  limited 

applicability. 


m- 


A  MORE  GENERAL  APPROACH 

The  purpose  of  this  paper  is  to  develop  a  procedure 
to  test  the  hypothesis  H  :  d.  =  f   Yi  against  various 
alternate  hypotheses  (as  yet  unspecified)  under  the  four 
assumptions  listed  above,  and  no  others e 

In  Chapter  2,  we  develop  a  graphical  representation  of 
the  observations  (j.,k  ),  derive  the  likelihood  ratio  test 
critical  region  characterization,  present  an  exact  test  for 
HQ:  d.  =  f  against  EL  :  d  4   f   (or  d  >f  )  for  a  given 
individual,  and  explore  three  apparently  unfeasible 
approaches. 

Development  of  a  general  two  stage  test  procedure  is 
given  in  Chapter  3»  and  in  Chapter  4  we  adapt  the  two  stage 
test  procedure  to  the  prototype  situation,  present  a  de- 
tailed case  study,  and  make  some  comparisons  with  the  earlier 
approaches „  Ae   conclude  with  the  derivation  of  an  approxi- 
mate, but  rapid  procedure  which  may  be  useful  in  some  cases. 
Computer  codes  necessary  to  implement  the  two  stage  procedure 
are  given  in  two  appendices e 
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CHAPTER  II 

BACKGROUND  AND  "CONVENTIONAL"  APPROACHES 

The  user  of  "conventional"  approaches  to  testing,  the 
hypothesis  H«  :  d,  =  f   \/i  against  some  alternate  meets  with 
difficulty  due  to  the  need  to  specify  a  value  of  d.  (or  f  ) 
as  will  be  shown  below 0   Before  doing  so,  however,  it  is 
convenient  to  introduce  a  graphical  approach  to  the  problem, 
derive  the  likelihood  ratio  test,  and  consider  a  special 
case  (n=  1  ). 

A  GRAPHICAL  REPRESENTATION 

For  ease  of  communication,  it  seems  advantageous  to 
utilize  a  graphical  display  of  the  data  such  as  shown  in 

Figure  18  where  the  observations  (j.k  )  are  normalized  (by 

i  i 

dividing  by  m)  and  plotted  in  a  unit  square.   Intuitively, 

one  feels  that  if  d,  =  f,  Vi»  then  the  points  (j. /m,k, /m) 

ii  i  i 

should  all   lie  near  the  line    1      =  k    ,    whereas   if  d^,  >f      Vi 

i    i  i   i 

then  the  points  ( j  . /m , k*/ni )   should  tend  to  lie  to  the 

right  of  the  line  d„  =  f , ,  etc0   In  a  sense,  the  test  of 

a.,,%   d  ss  f   Vi  against  some  alternate  hypothesis  is  a  test 
u   1    i 

of  "closeness"  of  the  points  (J  /m,K./m)  to  the  line  j.  =  Is.   , 

11  ii 

This  concept  will  be  helpful  In  the  succeeding  discussions. 

THE  LIKELIHOOD  RATIO  TEST 

It  would  be  desireable  to  have  a  likelihood  ratio 
test,  since  the  Neyman-pearson  Lemma  then  provides  that  for 
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7^73" 

Figure  1 
Graphical  Representation  of  Data 


1.0 


simple  hypotheses,  the  test  is  best  of  its  size.   Let  us 

consider  specifically  the  test  of  H0 :  d^  =  f    i  against 

H,  :  d  4   f.    1.   Let  v  be  the  vector  ( j^,  ,  J_  ,k0, . . .  ,  j  ,k  ). 
ill  -L  \     c.     e.  n  n 

By  independence  the  likelihood  of  v  is  given  by 

nfmm  U  k<  m-j.  m-k    \ 

L(v)   =     7TJ  (^    )(£    ){d1)    1{t1)    ^l-d^  l(l-f1)         1  j     (1) 


The   parameter  space  <n»is   given  by 
A  =  j^VV5    0  <  di   <  1,      0  <   f1  <  1,    1=  1,2,...  fn  > 


(2) 
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while  the  null  space  CO  is  given  by 


CO  =  |(d1,fi);  0<  d1<l,   O^f^l,   di=f1,  i=  1,2,...  ,nj. 

(3) 

It  is  convenient  to  operate  on  Ln  L  rather  than  L  directly 
so  we  maximize 

LnL  -  Z     1  j.ln  d,  +kln  f,  +  (m-j„  )ln(l-d.  )  +  (m-k  )ln(l-f  )  \ 

is-i  L  ■*•      *        1     -1     *•     i  -* 

(*) 

over  the  nu.ll  and  parameter  spaces e   Adopting  the  convention 
0**Ln  0=  0,  (that  is  extending  the  function  x  •  ln  x  to  0  by 
continuity),,  and  maximizing  yields 


maxT  T 
w  LnL 


n  f  j,+k,  j,*k  1 

(*)-J  t(J14^)la(-^)+(2«-J1.k1)la(1^4ii)J 


(5) 

and 

k   1* 

(m-k.  )ln(l  -  — )  f  .  (6) 

l        ml 

The  logarithm  of  the  likelihood  ratio   (v)  =  m£f   L(v)/^xL(v) 
is  given  by 


*  up  to  additive  constant  terms 
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.     n  r  Ji+ki  j^k- 

Ln  A(v)  =  ^|ji+k1)ln(-~-i)  +  (2a-j1-k1)ln(l-  •J~-i) 

j .         k.  Jj  k   "1* 

-  j  In  (-A)  -  k  ln(-i)  -  (m-J .  )ln(l  -  — )  -  (m-k  )ln(l--i)[. 
i     m     l    m        i       m       I       m  J 

(7) 
and  we  may  characterize  the  test  "by  rejecting  when  Ln  X(v)<C 
(constant).   The  boundary  of  the  critical  region  is  then  the 
hypersurface  in  (2n  -v  1 )  dimensions  defined  by  Ln  X(v)  =  C, 
which  is  difficult  to  visualize.   However,  by  the  indepen- 
dence among  pairs  of  observations,  contour  lines  of  Ln  >s(v) 
in  any  of  the  n  planes  defined  by  j.  =  0,  k.  =0,  i^l  for 
1  =  l,2,...,n,  will  be  identical,  and  as  shown  in  Figure  2. 
It  may  be  noted  that  Ln  X(v)  could  be  quickly  evaluated  by 
plotting  the  points  ( j^/m,k./ni) »  i  =  l,2,...,n  on  Figure  2, 
and  summing  the  values  of  the  contour  lines  on  which  they 
fall.   This  reinforces  our  original  intuitive  notion  of 
rejecting  EL  if  too  many  points  fall  "far"  from  the  diagonal 

h  -   V 

For  a  one   sided  test,  say  EL:  d^  > f   \/i,  the  likeli- 
hood contour  line  plot  corresponding  to  Figure  2  is  shown  on 
Figure  3. 

It  appears  we  are  now  ready  to  complete  the  procedure. 
We  need  only  find  C  such  that  p[Ln  X(v)  <  C]  =  0C   where  o(is  the 


*  up  to  additive  constant  terms 


1? 


-L   •  V/ 

LnX=  -  4    > 

\/m 

/nxzr-y7 

^6i\=  - 1 

/Lnx=  -y 

0.0 

0.0 


Jj7m 


1.0 


Figure  2 
Contour  lines  of  Ln   (v)  In  the  plane 
defined  by  J  =0,  k  =  0,  i^l(two  sided  test) 

desired  si^e  of  the  test.   But  here  is  where  the  difficulty 
arises.   The  conventional  approach  to  finding  pj_Ln  >(V)  <  Z J 
oald  he  to  note  that 


?[Ln  X(v)  =  Cv]  =  Z    p[V  =  v  ]   where  (8) 

scLr       s 

Iv  =  [s;  Ln  >(vc;)  -  CV  j  ,  and  so  if  VQ   =  |^ ;  Cv  <  Cj 
PLLn  A(V)  <  C]  =   £    2   P[V  -  vsJ.  (9) 


we  nave 
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1.0 


1.0 


Figure  3 
Contour  lines  of  Ln   (v)  in  the  plane 
defined  by  J.=0,  k.=0,  i^  1  (one  sided  test) 


Both  P  and  1^  are  readily  found  for  all  G  and  *r  ,  but  the 
c 

probability  that  V  =  v  cannot  be  specified,  for  assumption 

S 

2  states  only  that  J  =  b(m,d  )  and  K.  =  b(m,f  )  with  d  and 
f  unspecified.   This  inability  to  complete  the  relationship 
betx\reen  the  size  of  the  procedure  and  the  critical  value  C 
renders  the  "conventional"  approach  useless. 


SOMS  "DIRECT"  ATTEMPTS  TO  FORMULATE  TESTS 

Three  comparatively  straightforward  attempts  to  circum- 

4 

vent  the  lack  of  specific  distributions  for  J  and  K.  were 
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made.   A  brief  summary  of  each  with  an  explanation  of  why  it 
failed  follows. 

The  first  attempt  involved  the  transformation 


-  £  -  ^  •  ** 


oh  is  a  projection  of  the  points 


( J4  A1  j^Vm)  onto  an  axis  orthogonal  to  the  line  j.  =k  ,  as 


-■Yi 


u(l/2,  3A)  =  -^l/32 


u(.6,  A)  =  -12AQ 
u(l,  .8)=  -VFlO 


Figure  4 
The  Angular  Transformation  u(J  /m,k  /m) 
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illustrated  in  Figure  k.      For  example,  the  point  A:  (1/2,3/^) 
is  transformed  into  u( 1/2,3/^)  =  ^1/32.   Intuitively,  one 
would  hope  to  find  a  good  procedure  which  rejects  H  if  too 
many  of  the  u,  fall  close  to  the  end  points  of  the  interval 
(  -  yi/2,  ,/l/2  ).  The  details  of  why  such  a  procedure  is  not 
good  are  burdensome,  but  the  cause  is  easily  shown  graphic- 
ally by  the  points  B  and  D  in  Figure  4.   Note  that  both  B 
and  D  map  to  the  same  point  E  on  the  U  axis,  yet  B  represents 
a  Ln  X  value  of  -  1,  while  D  falls  on  the  Ln  X  =  -  4  contour. 
Thus  if  C,  the  critical  value,  was  -  2,  say,  point  B  should 

cause  acceptance  of  H  .  while  point  D  should  cause  rejection, 

0 

yet  given  only  point  E  on  the  U  axis,  we  cannot  determine 

whether  it  is  the  image  of  B  or  D,  hence  this  approach 

fails.   (More  complicated  transformations  along  the  same 

general  lines  were  also  attempted,  but  without  success. ) 

n 
The  second  attempt  involved  use  of  the  sums   2  j*  and 
n  1=1  x 

2  k.  as  test  statistics.   Again,  the  details  of  the  break- 
i=l  1 
down  are  burdensome,  and  a  graphical  argument  demonstrates 

the  problem  quite  clearly „   As  an  example,  we  shall  consider 

two  different  experiments,  each  with  n=2,  with  results 

plotted  on  Figure  5»   The  points  A  and  B  are  from  the  first 

experiment,  the  points  A'  and  B*  from  the  second.   The  sums 

n         n 

2  j.  and  2  k,  are  the  same  for  each  experiment  (1.1  in 
i=l        i=l  x 
all  cases),  yet  clearly,  the  first  experiment  (points  A  and  B) 

should  tend  more  toward  rejection  than  the  second  (points  A' 

n         n 
and  Bf).   In  shorts   2  j.  and  2  k   are  not  sufficient 

15=1    X  1=1    1 


statistics  for  d.  and  f .  ,   i  =  l,2soeo, 
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n9 


Figure  5 
Two  divergent  experiments  with  equal  data  sums 

Discussion  of  the  third  "direct"  attempt  is  best 
deferred  until  the  end  of  the  next  section,  which  establishes 
some  necessary  background  and  notation. 

A   SPECIAL   CASE    (n=  1) 

Then  only   one   operator  is   used   in  the   experiment,    the 
procedure   known  as   Fisher fs   Exact   Test   for  Percentages   is 
applicable    .      A  brief  derivation   follows.      The   test   of 
interest   is     now 


I    :    d^    =   f     against    (say)   Hi    d„  > f  „ . 

Oil  111 


ZUr 


Now  under  HQ ,  and  by  independence 


PCJt  =  i±,\  =  \l   =  P[J1  =  J1]  .    pEk^^] 

j-i  +  k-,        2m  -  J1  -  k- 

mlm!  d1  x    ■"■  (1  -  d1)         1 

=      J1!(m-j1)|k!  (m-k^  )!    ~" 


(10) 


But   P[Ji  =  J1»    Kl  =  ki^    is   e(luivalent   to  P'LJi  =  ^i'J1  +  Ki  =  <3i+ki^ 
At   this  point   it   is   convenient   to   introduce   the  random 

variable   L    =  J    +  &.    so  that  we  have   p£j    =  j    ,K  ]   equivalent 

J.       J»       J.  J.       X     X 

to  P[J1  =  j  ,L  =1.1  where  1  =  j„  +  k  .   Equation  10  may  now  be 
u  1   1   1    1         11    1 


written 


11        2m  -  11 
mlmfd.  J"  (1  -d,  ) 


and 


P^Jl°;ll'Kl  =  ki:l=31,(l1-J1),(m-"j1).(m+31-l1)l   (11) 

we  note  that  we  must  have  0<j  <m,  0<(]1<l<2m.   Now 
(still  under  H  ),  the  random  variable  L*  is  the  sum  of  two 
independent  identically  distributed  binomial  random  variables 
and  so  is  binomially  distributed  itself.   Specifically, 

L-  =  b(2m,dj  ) ,  so 

11        2m  -  11 
(2m) !  d1    (1  -d-  )     1 
p[L  =1  ]  =  — 1—— — •  (12) 

By  the  definition  of  conditional  probabilities 

P[J,  =  J«  ,L-  =1  ] 

^■'ilvv     p^-u      •  (13) 
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Substituting    (11)    into    (12)    and    (13)   yields 


P[J1  =  J1|L1  =  11]  = 


11  2m  -  11 

m!m!d1         (1  -  d    ) 

hl  (11  -  J  1  )  !  (m  ~  Jx  ) !  (m+  jt  -ljj! 


1      °iri  13  2m  -  lt 


(2m)!dj    J"    (1  -d^ 


(2m -11)!    lx! 


mlm!(2m-l1)l    1%  !  (1^ 


(2m)SJlS(l1  -  ^JKm-  J1)!(m+  j3  -  ^  )  ! 
Rearranging  appropriately  yields 


11 


which  is  the  hypergeometric  distribution  with  first  parameter 
(population  size)  2m,  second  parameter  (sample  size)  l*,  and 
third  parameter  (proportion)  1/2. 

vfe   now  have  the  distribution  of  the  number  of  true 
detections  J^  given  11  total  indicated  detections,  and  most 
importantly,  it  is  independent  of  d.  and  f .  .   Let  us  now 
develop  a  likelihood  ratio  test  of  HQ:  d.  =  f .  against  some 
as  yet  unspecified  alternate  hypothesis.   Letting  X( j -•  1 1-,  ) 
denote  ***   L(  jj  ^  )/?f X  L(jjl1)  where  co  =  [^€[0,1];  ^  =  f  J, 
/To  =  )  d1;0  <  d1  <  if,  and  L(JjJl-)  is  the  conditional  likelihood 
of  j.  given  1.,  we  have 

PCJl-Jlfcl-'ll.H,,]  (l6) 


AlJl|1l)"P[J1=  Jl|Ll  =  1l'Hl^ 
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Now  under  H< 


*LJi  =  h>h  =  1i\\l 


and 


li-j 


m~^l 


m+j.-l. 


m!m!d1  J"  f^  l      i(l-d1)     (l-f-j)      X 


J]  !(1l""Jl)3(m";3l)!(ni*t"Jl  "  11)! 


(17) 


pELl =  11 


^rm!m!d1a(i-dl)^f1^    (i-fl)    ;   1 

1    ^-1"    a!  (m-a)  !  (1-,-a)  !  (m-l.+a) !         I 
a=  a^  i        1  -J 


where  a  =  max  \o  ,  L-mr  and  b  =  min  jm  ,1-  >  . 
Substituting  (17)  and  (18)  into  (13)  we  have 


(18) 


P[jl  =  jl|  1^  =  1^]  = 


l/^iKl^JiJKm-J^Km-l^j^l} 


I 


|f^i=3JJ 


a~Jl 


a=a  a  !  (m-a  )  !  (1-,  -a  ) !  (m-l«  +a  )  ! 


And  substituting  (1*0  and  (19)  into  (16),  we  obtain 


(19) 


X(J1|i1)  = 


m!m!  (2m-lj  )  !l^  !  ^ 


Lfi<i-di>J 


(2m)! 


a=a 


^  a  !  (m-a  ) !  (1.  -a  ) !  (m-1-.-Ki ) 


(20) 


Let  us  now  consider  a  specific  alternate  hypothesis,  say 


E.  :  d  >  f .  .   We  then  have 


fid-^) 


>1  and  XU-Jl,)  is  mono- 


tonically  decreasing  In  J^.  Since  we  would  reject  HQ  if  the 
observed  test  statistic  X(Ji|l  )  ls  sufficiently  small,  say 
less  than  some  critical  value  G,  we  can  perform  an  equivalent 
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test  by  rejecting  HQ  if  the  observed  value  of  j1  is  greater 
than  Cf,  where  j.  >C*  is  equivalent  to  X(jJl1)<C.  vie 
note  also  that  this  test  is  uniformly  most  powerful  for 
EL  since  the  procedure  would  be  the  same  for  all  cL  >  f  . 
The  size  of  the  procedure  is  P[j-,  >C*  l^H  ],  which  can  be 
readily  computed  using  equation  15  >  and  the  power  at  a 
specific  point  (d. ,  f  )  can  be  computed  using  equation  19. 

For  a  two  sided  test,  i.e.  FL  :  d  ^  f .  ,  a  similar 
development  shows  that  the  optimum  rejection  region  is 
characterized  by  the  rule  "reject  HQ  when  j.  <A  or  j.  >m-A" 
where  A  is  such  that 

P[JX  <a|ii,H0]  +  p[J1  >m-A|l1,H()]  =  2P[J3  <A|l1,H0]  =  a 

(Of  course,  not  all  values  of  a€[0,l]  are  available.   ,tfe 

assume  that  a  is  chosen  from  among  the  realizable  values. 

Randomization  to  achieve  arbitrary  sizes  may  be  used  but  we 

shall  omit  the  details.   To  include  such  a  consideration  here 

would  only  add  complication  without  adding  substantive 

information.  ) 

It  now  seems  natural  to  extend  the  above  result  to 

cases  where  n>l.   This  was  the  third  "direct"  attempt 

alluded  to  in  the  previous  section.   Specifically,  it  was 

hoped  that  the  families  of  probability  functions 

n       n  n  n  _ 

P[  2  J,    2]  1,  ,Hn]  and  P[  Z  J.    2  1,  ,H1  ]  could  be  obtained 

1=1   l   i=l  1      0        i=l  l  '  i=l  x   x 
and  used  in  a  fashion  analogous  to  the  preceeding.   However, 

whereas  the  function  P[J1  1 1 - , Hn J  was  independent  of  d.  and 
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n 


n 


f\  ,  such  is  not  the 


case  for  P|~  2  J.  Z     1  ,H  ]•   The  need 
i=l  1   1=1   i   ° 
to  specify  d.  and  f  ,   i=l,2,...,n,  which  is  not  per- 
missible under  our  null  hypothesis,  forced  the  termination 
of  this  approach. 
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CHAPTER  III 

THE  GENERAL  T/JO  STAGE  TEST 

'•rfhile  directed  at  the  prototype  experiment,  the  follow- 
ing discussion  has  somewhat  broader  applicability  than  to 
only  the  specific  situation  discussed  heretofore  in  this 
paper.   For  that  reason,  we  will  adopt  a  slightly  more 
general  notation  for  this  chapter  only.   tfe  will  consider  a 
test  of  the  hypotheses 

H~:  £^  =  9,.  Vi   against 
0    fi    di 


V 


efi  -  edi  I  =  Ai  Vl 


and  suppose  that  we  have  available  tests  T-,  thru  T  of  the 
respective  hypotheses 


H   :  9fi  =  9di   against 


Hu: 


fi    di 


=  *1 


for  each  i;  i  =  l,2,..0,n.   Je  shall  denote  the  size  and 

power  of  T.  by  a.  and  rr.  ,  respectively. 

It  seems  reasonable  that  the  outcomes  of  each  of  the 

tests  Ti  thru  T   should  contain  some  information  about  Hn 
In  0 

and  IL  .  Ae   will  develop  a  procedure  by  which  the  results 

of  T,  thru  T   (hereafter  called  the  first  stage  tests)  can 

be  tested  in  order  to  make  inferences  about  E     and  HL  . 

This  test  on  the  results  of  T1  thru  T  will  be  called  the 

1       n 
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second  stage  test,  and  the  complete  procedure  (applying  the 
first  and  second  stage  tests)  will  be  called  a  two  stage 
test.  tie   begin  with  a  special  case  before  developing  the 
general  procedure. 

IDENTICAL  FIRST  STAGE  TESTS 

Suppose  that  H1  is  of  the  form  H-  ;  I  ©-..  -  © , ,  I  =A  Vi. 
1  1  I  fi    di  I 

Then  the  respective  alternate  hypotheses  for  the  first  stage 
tests  will  be  H-,:  I  ©fi  -  © , .    =^.  for  each  test  T.  , 
i=l,2,...,n.   Now  if  T  thru  T  are  identical,  that  is 
have  the  same  operating  characteristic  curve,  and  we  apply 
each  at  the  same  size  (a*  = a  Vi)»  they  each  provide  the 
same  power  (n^(A)  =  tt(^)VI).   Now  let  us  define  the 
random  variables  X-  ,X? , . . . ,  X  by 


xx 


To   if 


T,  results  in  acceptance  of  H  . 


1   if  T.  results  in  rejection  of  H  . 
i  oi 

n 


Under  Hn  all  of  the  H   are  true,  so  if  I  =   2  X.  then 

d    u  0i  i=l  x 

Y  =  b(n,a).   Similarly  if  IL  is  true,  then  all  of  the  & 

are  true  and  here  Y  -  b(n,rr(  A))»  Let  us  denote  the  (general) 

distribution  of  Y  as  b(n,p).   Then  if  we  test 


H':  p  =  a  against  H*  :  p  =  n(A) 

we  have  actually  tested  Hn  against  H  since  Hi  is  true  if 

and  only  if  the  EL,  are  true  for  all  i  and  HA  is  also  true 

Oi  0 

if  and  only  if  all  the  HQ1  are  true0   In  similar  fashion 
H'  implies  H>  and  conversely. 
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Using  the  well  known  likelihood  ratio  test  of 
Hq:  P  =  0(  against  H*  :  p  =  7T(  A  ) ,  the  procedure  will  be  to 
reject  B.     if  the  observed  value  of  Y  exceeds  a  critical 
value  y*  where 

PCiy  I  HZ]-     2   #  (2)ak(l-a)n"k  =  a»       (1  ) 

0   k  =  [y]*+i 

and  the  symbol  |_-J  denotes  the  greatest  integer  function. 
The  test  of  HA  against  H*  is,  of  course,  the  second  stage 
test.   Now  since  we  can  substitute  H0  for  H'  in  the  left 
hand  expression  of  equation  (1),  we  see  that  a'  is  the  size 
of  the  two  stage  procedure.   Likewise,  the  power  of  the 
second  stage  test  is  given  by 


PCY>yf  |H']=    S     Cg)  n(A)k(l-rr(A))n"*k=TT'   (2) 
1    k=[y]  +1 


and  here  too  we  can  replace  H]  by  H*  ,  so  n*  is  the  power  of 
the  two  stage  procedure.   It  is  interesting  that  a'  (the 
size  of  the  two  stage  test)  may  be  set  independently  of  a, 
the  common  size  of  the  first  stage  tests. 

An  examination  of  equation  2  suggests  that  the  nota- 
tion tt*  =  n'(A»  a'»  tt(A>&))  should  be  used  to  emphasize 
the  functional  dependencies  involved.   ^e  will  write  the 
equivalent  form  n*  =  rr'(A,  a*,  a).   Thus,  while  a'  may  be 
set  independently  of  a,  the  choice  of  both  a  and  a'  in- 
fluences tt'.   Generally  speaking,  we  will  be  given  (or  be 
willing  to  specify)  A  and  p( '  so  our  concern  would  be  to 
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choose  an  a  that  maximizes  n'(^>  a'»  a )  •   This  requires 
solution  of  the  program 


n    n  -,k  n  -  k 

Maximize  tt'  =     £   (£)  [n(A,a)]   [l-n(A,a)] 
K=[y»]  +1 


n         v       n  —  k 
Subject  to:     2  „   (P )  a   (1  =  a)    '  =  a1  ,  0  <  a  <  1. 
k=[y»]*+l  k  "  " 

If  the  appropriate  OC  Curves  for  the  second  stage  test  are 

available  (that  is  if  sufficiently  extensive  binomial  tables 

are  available),  a  simple  search  procedure  may  be  used  to 

find  the  solution.   Otherwise  it  appears  that  some  iterative 

technique  will  be  required.   In  any  case  we  will  not  consider 

this  problem  further  in  this  paper,  except  as  it  applies  to 

the  prototype  case  (Chapter  4-). 

Two  further  observations  are  appropriate.   First,  the 

two  stage  test  may  be  one  sided  if  one  sided  first  stage 

tests  are  available.   Similarly,  either  or  both  of  H  and 

H  can  be  made  composite  provided  the  corresponding  H„. 
"1  Oi 

and/or  H„ .  can  be  made  comDOsite  and  with  a  slight  modifi- 
li 

cation  in  the  definition  of  power.   The  derivation  in  such 
cases  requires  only  minor  modification  of  the  foregoing. 
Second,  there  is  no  assurance  that  the  two  stage  test  repre- 
sents a  uniformly  most  powerful  test,  even  when  the  first 
and  second  stage  tests  are  in  themselves  uniformly  most 
powerful.   It  appears  that,  in  general,  the  two  stage  test 
may  not  be  most  powerful,  although  its  power  characteristics 
seem  to  be  good.   „-/e  return  to  this  question  in  Chapter  k. 
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GENERAL  FIRST  STAGE  TESTS 

We  now  relax  the  requirement  that  the  first  stage 
tests  T.  ,T2, . . .  jl^be  identical  and  assume  only  that  Oi    and 
7T^(rf  )  are  specified  for  each  T^ ,  i  =  l,2,...,n.   We  will  con- 
sider testing  the  hypothesis 

Hn:   0   =0   Vi   against   H,  :  |  9   -  9  .  I  =  £>.  Vi 

0    fi   di  1  •  fi   di  '     i 

for  which  the  appropriate  first  stage  tests  are,  for  each  i, 

H  .  :   0...  =  0,,   against   H„  .  :  I  ©^  -  0   I  =  A. 
oi    fi   di  11  I  fi   di  I      i 

If  some  of  the  first  stage  tests  are  identical,  we  will 
group  them  together,  so  we  will  have  (say)  k  groups  of 
identical  tests  (that  is,  identical  within  a  group)  to 
consider.   (For  example,  if  no  two  tests  T.  ,  T.;  i^j  are 
identical,  then  k  =  n,  whereas  if  all  tests  are  identical 

k=l.)   Let  0.    denote  the  number  of  (identical)  tests  in 

th 
the  j    group,  j  =  l,2,...,k,  and  relabel  the  a  '  s  and  tt  '  s 

so  that  a.  and  n   represent  the  common  size  and  power, 
J      J 

respectively,  of  the  0.   tests  in  group  j .   Now  using  argu- 

j 

ments  identical  to  those  in  the  previous  section,  we  have  as 
an  equivalent  test  of  H_  against  H1 ,  the  second  stage  test 

HI:   pw  =  a  against  Hi:  prt  =  tt. 
0    0  1        0 

Here  p.  denotes  the  k  component  vector  of  success  prob- 
ability parameters  in  the  joint  distribution  of 

Y  =  (Y  . Y,.,...,Y  )  where  Y.  is  the  number  of  rejections 
12      k         J 


3^ 


(of  H   )  from  the  tests  in  group  j.   (Thus  I  cjO.l, . . .  ,0  }.) 
Oi  j  v-        j ) 

The  vectors  a  and  tt  denote  (a.  ,cu,...,a  )  and  (tt  ,tt  , . . .  ,tt  ), 

1      c  k        12      k 

respectively.   Now  if  a'  is  the  size  of  the  second  stage 
test,  we  have  p  ^rejection  of  Hi  |  H'l  =  P  [rejection  of 
H'  I  H  I  so  a*  is  the  size  of  the  two  stage  procedure  as 
well.   Similarly,  the  power  of  the  second  stage  test  it', 
is  p  ["rejection  of  HJ  |  H •  J  which  is  equivalent  to  p  T 

rejection  of  ill  |  H..J,  so  n'  is  the  power  of  the  two  stage 
procedure  also. 

Let  us  now  get  the  second  stage  procedure.   By  inde- 
pendence 


k 


j  =  l     J    J 


(3) 


Under  H* ,  Y  is  the  number  of  "successes"  in  0  trials  at 
0   j  j 

constant  probability  a . ,  or  Y.  =  b(0 . ,a  ),  j  ■=  l,2,...,k. 

d       ^ 
Similarly  under  H*  ,  we  have  Y  =  b(0 .  ,tt  ),  j  =  l,2,...,k. 

1  0      J   j 

Then,  by  equation  3>  under  H* 


k 


LQ(y)  =  P[Y=y  |  HJ]  =    TT    Nyj  )  ct.'j    (l-c^) 


*<•        v.    ..  -Vyj] 


(4) 


and  under  H* 

1 


Ld  (y)  =  P[Y=y  |  HJ]  = 


k 

7T 
3=1 


0 


y 


(*J)   n  *j    (1-tt    )   J 
^y^       j  j 


0.  -y 


■} 


(5) 
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Thus  the  likelihood  ratio  X(y)  =  L  (y)/  LAy)   is  given  by 


X(y) 


K 
C  -  7T 


a,  (1  -Tt,)JJ 

J vL 


n.  (1  -a.) 

lj     y 


k 


where  the  constant  C  is  7F 

have  n.  >a  Vj  ,  hence, 
0  ~  j 


1  -  a 


1  -TT 


0 


(6) 


In  general  we  will 


a.  (1-tt  ) 

— - — — -.   <  1  Vj ,  so  X(y)  is  monotonically  increasing 

n   ( 1  -  a  )  ~ 

J  ^ 

as  a  function  of  any  component  y.  of  y.   Characterization 

j 

of  the  optimum  rejection  region  is  best  accomplished  by 
considering 


^    k 

Ln  X(y )  =  Z     y  .  [lna.  -  ln(l  -  a,  )  +  ln(l  -  n1  )  -  In  n.]  +  InC.  (7) 
\  —  \      J     J         J  <J       J 


rfe  will  reject  H'  when  ln  >sCy)  is  less  than  some  critical 

value,  sa.ymJ\^,      Since  the  quantity  in  the  brackets  £•]  in 

equation  7  is  negative  for  all  j ,  the  general  tendency  will 

be  to  reject  HI  when  several  or  all  of  the  y.'s  are  ,f^rge' 

(near  0.),  an  intuitively  appealing  result, 
j 

Specifically,  we  may  label  all  possible  outcome  vectors 

y  so  that  ">\(yl  )  <  X^  i)>  1  =  1 ,2,. .  .  ,v-l,  where  v  is  the 
l       1+1  k 

number  of  different  y's  Dossible.   (v =  2  [fi  •  +  l]« )   Then 

j=l   J 
the  critical  region  consists  of  the  set  of  vectors 

r 
where  r  is  such  that 


[t15y2!...,yr] 


Z   P[Y=y.|  H']  -  a'. 
1=1      1   u 


(ie  again  assume  a*  is  chosen  so  that  it  is  a  realizable 

r    _^ 
value.)   The  power  n'  is  then  2  F|_Y="y  I  H?  J. 

1=1 


1 
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There  are  several  schemes  for  reducing  the  work  re- 
quired to  produce  the  critical  region  (or  equivalently ,  the 
acceptance  region).   The  most  straightforward  is  to  find 
~y     and  compute  p  [Y="y«  |  H!j.   If  P  [?=  y.  |  H']  <  a* ,  f ind  "y 

-L  J.     \J  g  J.     \J  £ 

and  compute  p  [Y="y  I  HlJ.   If   S  P  [?=  y,  I  H'  ]  <  oi«  ,  find 

s     ~L    ^  i==1 

y~  etc.  ,  until  L     p[Y=y.|H,]  =  cc»,at  which  point  s  =  r 

J  i=l        ll  o 

and  the  critical  region  is  defined.   An  identical  procedure 

may  be  used  to  define  the  critical  region  for  a  test  of 

-a 
specified  power  by  using  the  distribution  of  Y  under  Hi 

rather  than  HI.   An  even  faster  method  (in  most  practical 

cases)  is  to  start  by  finding  y  and  working  "down"  to  the 

required  size  (or  power)  in  a  similar  manner. 

A  computer  code  (Appendix  B)  has  been  written  which 
can  complete  a  problem  with  n=20,  k=5  (5  different  first 
stage  tests)  in  less  than  two  minutes  on  a  medium  sized  third 
generation  computer.   This  program  will  give  the  acceptance 
region  and  power  of  a  test  of  some  given  size  or  if  desired, 
will  print  out  all  or  a  portion  of  the  cumulative  distri- 
bution function  of  Y  under  both  HQ  and  BL» 

In  this  test,  as  with  the  identical  first  stage  tests, 

there  is  some  optimal  choice  of  the  values  for  a  ,  J=l  ,2, ...... fk 

j 

which  will  maximize  n'  for  a  given  size  a1.   The  appropriate 
program  is 


r    _^ 
Maximize  n1  =  2  P  [Y=y,|n(cx)] 
13=1       ± 
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Subject   to:        S     P[Y=yJa]  =  a1    and     \(t  )  <    \(y*      ) 
i=l  ll  1    ~  i+1 

for  all   i,    i=l,2,.co,v-l. 


This  is  a  challenging  program  indeed  and  we  shall  not 
discuss  the  problem  here,  except  to  state  that  in  the  proto- 
type situation  it  was  possible  to  obtain  powers  approaching 
those  available  under  the  assumption  of  equal  performance, 
using  an  ad  hoc  procedure  (Chapter  4). 

As  a  final  comment  we  note  that  one  sided  tests  and 
tests  with  composite  hypotheses  may  be  made  if  appropriate 
first  stage  tests  are  available.   The  derivation  of  the 
specific  two  stage  test  for  such  cases  can  be  completed  in  a 
manner  analogous  to  the  foregoing. 
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CHAPTER  IV 

ADAPTING  THE  TWO  STAGE  TEST 
TO  THE  PROTOTYPE  SITUATION 

We  now  consider  the  application  of  the  two  stage  test 
to  our  prototype  testing  situation,,   We  wish  to  test 


HQ:   dl  s=  f   Vi,  against  (say) 

Hl:   di  "  fi  =  ^i  Vi'  i=1»2»-"»n- 

The  use  of  the  two  stage  procedure  is  facilitated  by 

specification  of  the  A  • s  in  such  a  manner  that  Fisher's 

Exact  Test  can  be  used  as  a  first  stage  test.   Thus  we  will 

specify  A,  so  that  tt.  (21.)  is  a  constant  for  all  d  and  f 

which  satisfy  d,  -  f .  =  A  . 

i   1     i 

Let  us  consider  equation  19  of  Chapter  II,  which 
may  be  used  to  compute  the  power  of  Fisher's  Exact  Test. 

We  consider  now,  however,  the  application  of  Fisher's  test 

th  th 

to  the  i   individual o   For  the  i   individual,  equation  19 

becomes 


£Ji  -  'i  I  \  -  VHi]  - 


b        d1(i-firr-J1J1!(l1-J1)|(ni-Jl)!(m+J1-l1)! 
ria      fi(l-d1)  r!(lfr)!     (m-nr)!(m+r-l1)! 

where      a  =  maxlO,    1     -m|,    and  b  =  min  W,    1    f. 


(1) 
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1.0 


FIGURE  6 
Some  Sample  Specifications  of  H 


li 


Let  us  specify 


1  -  f , 


11   1  i       [f i  -  (o-1)]  +1 

wnere  D  € (0, °«). 

A  graphical  interpretation  of  this  specification  is 
shown  in  Figure  6  for  several  values  of  D. 

Two  interesting  features  emerge  from  this  specification. 
First,  as  seen  in  Figure  6,  this  specification  for  tL,  tends 


kO 


to  satisfy  one's  intuitive  feelings  about  a  reasonable  form 

for  H« ,  .   That  is,  we  feel  that  the  difference  d.  -  f 4 ,  about 
li  11 

which  we  are  concerned,  can  be  relatively  large  for  f  at 
midrange ,  but  should  shrink  as  f  approaches  its  extreme 
values.   Second,  solving  d,  -  f .  =  (1-f,  )/  [f  (D-l )]   +1  for 

a.d-fi)  di(1~fi)         i  l" 


f  (1-d. ) 
l    l 


yields 


Ml-d,  ) 


=  D,  and  equation  1  may  be  written 


P[J1  =  Ji|Li  =  l1>H1> 


b 
-r=a 


[D] 


f-h  J^W^i^ 


-Y\  (l1-*T  )!  (m-r)!  (m+ 


m+T-l    )!  J 


The   power  of   the    (first   stage)   test   is   then 


(2) 


b 
1   J^JJ+a+1 


V-Jj_  jj_!  (li-^)!  (m-ji)!  (m+ji-li)| 


nr=a 


"^  !  (li--y)!  (m-v)!  (m+V-1  )! 


(3) 


where  j  '  is  such  that 


P[ji>JlWH0i] 


b 
Z 
J.=J'+a+l 


m     m 

(3i)(1r3i),. 


(?m) 

i 


(M 


Should  we  desire  a  two  sided  test  we  can  specify 

1-f, 


H 


li* 


d  -  f 
i    1 


[f.  (D-l)]""1  +1 


d,  (1-fi  ) 
Solving  for  — +-. =-r  in  this  specification  of  H«,  yields 

1^   i 
the  solutions  D  and  1/De   It  may  be  readily  verified  that 

equation  3  yields  the  same  value  of  tt  when  1/D  is  substituted 


kl 


for  D.   Hence,  the  power  of  this  (two  sided)  test  is  still 
given  by  equation  3  provided  the  value  of  i  •  is  set  by 


«'1  =  P[J1>3i|L1  =  l1,H01>P[J1<l1-J1|L1  =  l.,H01]  = 

b        <"?    ><1   -J    > 
2-  Z  Li      1  (5) 


j   =j'+a+l         Cn     ) 
1      J  i 


THE  TtfO  STAGE  TEST  FOR  THE  PROTOTYPE  CASE 

vJe   now  discuss  our  test  of  the  hypotheses 


H  :  d  =  f   Vi  against 
0   i    i 


H  :  d  -f,  =  — —±-. Vi,  D  €(0,— ). 

1   1   x   Cf^D-a)]"1^ 


The  first  stage  tests  T,  ,ToJ...,T  will  be,  for  each 

l   2      n 

1,  i=l,2,...,n,  Fisher's  Exact  Test  of  the  hypotheses 

H   :  d  =  f   against  HL  ,  :  d,  -  f .  =  ■= =j-i 

Oi   i    i  li   i   l   [f  (d-1)J  i+l 


Given  the  observed  values  1  =  (1  ,1. ,...,1  )  of 

1  c  n 

L  =  (L,  tL.|... »L  )  (total  indicated  detections  by  each 
12      n 

subject)  we  form  (in  general)  m  groups  of  results  by  the 
rule  "group  V  contains  all  results  where  1  =Tor  1  =  2m-Y% 
V=  1 ,2, . . .  ,n.  "  Because  Fisher's  test  is  the  same  for  1  =  *V 
or  lj_  =  2m-*V,  the  tests  to  be  applied  within  each  group 
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are  indeed  identical.   Continuing  in  the  notation  of  Chapter 
3,  the  number  of  results  (or  equivalent ly,  the  number  of 
first  stage  tests)  in  group  *V  will  be  denoted  0^  for 
v=  1,2,. . .  ,m.   Of  course  some  of  the  0^s   may  be  zero,  and 
while  the  derivation  of  the  two  stage  test  did  not  take 
into  account  that  possibility,  it  is  easily  shown  that  any 
number  of  the  0^' s  may  be  zero  without  affecting  the  pro- 
cedure or  its  power.   It  is  convenient  here  to  set  k,  the 
number  of  different  tests,  equal  to  m  and  allow  some  of  the 
0^,' s   to  be  zero,  as  necessary. 

We  may  use  the  computer  code  of  Appendix  A  to  compute 
the  available  values  of  av  and  n^.,  that  is,  the  possible 
non-randomized  sizes  and  corresponding  powers  of  the  (first 
stage)  tests  in  each  group.   We  then  choose,  by  a  method 
discussed  below,  a  size  a-r  for  the  tests  in  each  group  and 
apply  Fisher's  test  at  that  size  to  all  results  in  that 
group,  for  each  group  ***,  Ts  l,2,...,m.   The  computer  code 
of  Appendix  B  may  then  be  used  to  obtain  a  characterization 
of  an  acceptance  region  for  the  second  stage  test  at  the 
desired  size  a'.   Finally  we  form  the  test  statistic 
Y  =  (Y  , Y  , o.»,Y  )  where  Y>-  is  the  number  of  rejections  of 
H_. .  from  among  the  0-r  first  stage  tests  applied  to  the 
results  in  group   .   If  Y  is  in  the  acceptance  region,  we 
accept  H  ,  otherwise  we  reject  H  and  the  procedure  is  com- 
plete. 

It  is  interesting  that  the  power  of  the  two  stage  test 
just  described  is  a  random  variable.   This  is  true  because 


^3 


in  the  equation 
r 


tt»  =  £  P  [Y=y JiT(a)] 
1=1        x 


(6) 


the  values  of  ar  and  n-r(a>.)  depend  upon  the  realization  of 

L,  (or  more  specifically,  the  values  of  0^,  T«  1 ,2, . . .  ,m). 

^e  note  further  that  the  concept  of  expected  power  cannot 

be  applied  here,  since  our  assumptions  preclude  specif ica- 

tion  of  a  distribution  for  L.   (Indeed,  suppose  we  did 

assume  a  distribution  for  L.   Then  we  could  (in  theory) 

derive  the  distributions  of  J.  and  L,  -  J. ,  which  would  be 

i      i   i 

logically  equivalent  to  fixing  values  of  d  and  f  ,  violat- 
ing our  assumptions. ) 

However,  even  when  the  power  of  the  two  stage  test 
obtains  its  "poorer"  values,  it  still  has  good  power  for 
practical  purposes,  and  in  fact  that  power  approaches  the 
power  available  from  tests  made  under  the  assumption  of 
equal  performance  by  all  individuals.   *Je  next  consider  the 
choice  of  the  best  set  of  sizes  for  the  first  stage  tests. 

CHOOSING  FIRST  STAGE  SIZfiS 

As  noted  in  Chapter  3»  the  "optimal"  choice  of  first 
stage  sizes  &v,  IT-  1 ,2 , . . . ,m  would  be  given  by  a  solution 
to  the  program 


Maximize  n«  =  £  p[?="y  |  n(a)] 

1=1       x 


i±l+ 


Subject  to:    £  P[5=y  I  a]  =  #  •   and 
i=l       v 


My^)  lA(f1+1),  i  =  l,2,...,v-l 


m 


where  v  =  77"  (0.-+1)  ,  a  =  (ai,a   .  OJa  )   and 

i=l   «  *      m 

*(S)  =  (TT1  (a„  ),n2(a0),oo.,n  (a  )). 
1   1    *   2       mm 

tfe  have  been  unable  to  solve  this  program  in  general, 
so  we  offer  only  a  rule  of  thumb  for  choosing  the  sizes 
of  the  first  stage  tests „   This  practical  approach  has  been 
successful  in  the  cases  examined  thus  far0   The  rule  in- 
volves use  of  non-randomized  values  for  GVonly,  and  only 

assures  a  "near  optimal"  vector  aTT  =  (a,  ,a0  ,...,a   ) 

e  w    lw  2w      mw 

where  by  "near  optimal"  we  mean  that  only  minor  improvement 

in  the  value  of  tr*  (say  in  the  second  or  third  decimal 

place)  would  accrue  if  the  true  optimal  vector  a  were 

o 

found . 

The  first  step  involves  finding  an  upper  bound  for 

n*.   Consider  the  result  0„  =  0,v=  l  ,2, «, . «,  ,m  -  1 ,  0  =  n, 

^  m 

which  represents  the  most  "informative"  result  obtainable. 
Since  such  a  result  reduces  to  the  identical  first  stage 
tests  situation,  we  would  be  testing  the  second  parameter 
of  a  binomial  distribution,  that  is 


HQ:   p  =  am  against  H^  :   p  =  n  , 


^5 


The  best  choice  of  (a  ,rr  )  can  be  obtained  for  that  case 

m  m 

by  enumerating  the  powers  7T'  obtained  from  each  of  the 

available  (a-^,rr^)  pairs,  using  tables  of  the  binomial 

distribution.   Let  TT  •  be  the  highest  such  power  obtained 

for  the  desired  size  a'» 

Now,  using  the  computer  code  of  Appendix  B,  we  may 

compute  TT  '  for  the  vector  a   (a.  ,a0  ,.«,., a   )  where 
a  a   la  2a      ma 

a   ,  *r"=  1 ,2,  <, . .  ,m  denotes  the  largest  non-zero,  non- 

■va 

randomized  value  of  ar  less  than  or  equal  to  0.5»  For   a 

two  stage  test  of  size  o05,  if  tt  »  is  within  (say)  .02  of 

tt  • ,  it  seems  reasonable  to  accept  a  as  "near  optimal" 

since  any  other  choice  of  a  will  improve  n1  by  .02  at  most. 

If  tt  '  is  not  within  .02  of  tt  •  ,  there  is  cause  to  believe 
a  u 

it  is  not  "near  optimal"  and  a  new  vector  a,  should  be 

_*» 
tried.   No  firm  suggestion  for  choosing  a   is  given,  but 

the  following  observations,  made  from  the  experience  of  the 

case  study  to  follow,  are  given  as  guidance. 

Components  of  a  for  which  0r  is  large  seem 
to  be  most  sensitive  (that  is,  a  change  of  one  "step" 
in  available  ar  values  causes  a  relatively  large 
change  in  tt").   Components  of  a  for  which  0^  is  small 
(say  1)  seem  quite  insensitive,,   Thus  in  seeking  the 
optimal  a  it  would  seem  advisable  to  change  the  a^'s 
for  which  0^.  is  large  first „ 

2.  It  appears  that  each  component  may  be  "near- 
optimized"  separately o  That  is,  if  0  is  the  largest 
0-y value,  then  one  may  find  the  best  value  of  a  for 
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arbitrary  values  of  the  other  dy*  s.   Having  found  a 

"near  optimal"  value  for  a  .  say  a   ,  then  one  can 

q*    *   qo* 

find  a  "near  optimal"  value  for  a  ,  say  a   ,  in  the 

p'      po* 

vector  (a1,a0,ooo,a  ,  coo,a   ,  oo.)  where  a^  is  arbitrary 
i  d.  p      qo 

for  all  *r  except  p  and  q  and  0  is  the  next  largest 
0rV   This  procedure  may  be  continued  until  all  compo- 
nents of  a  have  been  "near-optimized".   Due  to  inter- 
actions between  components  of  a  the  resulting  vector 
may  not  be  optimal  but  the  case  study  experience 
provides  strong  evidence  that  it  will  be  near  optimal. 

While  the  foregoing  seems  to  suggest  a  lengthy  procedure, 

in  a  very  limited  set  of  test  cases  considered,  the  initial 

vector  a  was  "near  optimal"  and  no  further  work  was 
a 

necessary.   Thus  it  may  be  that  the  choice  of  a  is  not  a 
problem  in  practiceQ 

A  CASE  STUDY 

Our  prototype  experiment  was  based  on  an  experiment 
conducted  at  the  US  Naval  Postgraduate  School  in  the  spring 
of  19680   There  were  n =  21  subjects,  each  of  whom  made  8 
attempts  to  detect  actual  targets  and  8  attempts  to  detect 
non-existent  targets  (m=8)e   The  value  D=  3  was  used  in  the 
specification  of  H1 ,  making  the  hypotheses  under  test 

l-f« 

H   :      d    =  f    Vi      against   Hi  :      d    -  f    =  t~ — -=rr\ Vi. 

0    i   i  l  i   i   [2f  J  1+i 

The  experimental  data  is  given  in  Table  20 


^7 


TABLE  2 
Results  of  a  Prototype  Experiment 


Subject  Indicated     True    Subject  Indicated     True 
Number  Detections  Detections  Number  Detections  Detections 


(i) 

<V 

(J,) 

(i) 

(V 

(V 

1 

12 

5 

11 

2 

1 

2 

7 

5 

12 

3 

2 

3 

10 

5 

13 

10 

5 

4 

10 

i+ 

14 

11 

4 

5 

7 

5 

15 

7 

3 

6 

9 

5 

16 

14 

8 

7 

15 

7 

17 

12 

6 

8 

12 

5 

18 

10 

6 

9 

14 

8 

19 

7 

3 

10 

7 

3 

20 

7 

3 

21 

9 

4 
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A  near  optimal  vector  a=  (a,  ,a_  ,  0  e  <>  ,a0  )  was  arrived 

1   2      o 

at  using  the  procedure  discussed  previously.   It  may  be 
informative  to  reconstruct  that  procedure  at  this  point. 
The  (hypothetical)  case  0V=  0,  T  m  1 ,2 , . . . ,7,0  = 2l  was  used 
to  get  an  upper  bound  tt  '  on  the  power  for  size  a1  =  .  05o 
The  various  (ar»nr)  pairs  available  for  each  group 'V, 
T=  1 »2 » 8 « o ,8  were  obtained  using  the  computer  code  of 
Appendix  A,  and  are  reproduced  in  Table  3t  along  with  a 
listing  of  the  total  responses (1.  fs )  which  fell  into  each 
given  group  V  according  to  the  rule  given  earlier  (group  "V 
contains  all  results  l^^nTor  1^  -  2m  -V)  and  the  critical 
value  for  conducting  the  (first  stage)  test  at  the  listed 
size  a  when  the  total  response  is  1,0   tt  '  was  found  to 
be  .994  and  the  first  trial  vector  a  was  (.50, .23, .50, .28, 

61 

,50,o30,.50,u3l  )o   Use  of  a  yielded  n  •  =  .98401  which  is 

near  optimal.   The  first  stage  tests  were  then  applied  at 

_j*  . 

the  sizes  in  a  with  results  as  shown  in  Table  4.   The 
a 

vector~Y  =  (Y  » Y2,  *  „  „ ,  YQ  )  -  (0,2,1,0,0,1,3*0),  which  may  be 
read  directly  as  the  right  hand  column  of  Table  4,  was 
found  to  be  in  the  acceptance  region  and  HQ  was  accepted 
at  the  o05  level  of  signif lcance0   Note  that  the  computer 
output  from  the  trial  computation  for  a  includes  a  complete 
description  of  the  acceptance  region ,  so  further  compu- 
tation was  not  necessary. 

ADDITIONAL  RESULTS 

Although  not  mentioned  in  the  section  on  choosing  a 
near  optimal  first  stage  size  vector  a,  there  is  evidence 
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TABLE  3 
Available  (a^,  nr)   Pairs  for  the  prototype  Experiment 


Group  Number 

(av,  tt^) 

1. 

1 

Critical  Value  C 
Reject  for  J.  >  G 

1 

(.50, .75) 

1 

15 

0 
7 

2 

(.23,-53) 

2 

14 

1 
7 

3 

(.50, .83) 
(.10, .36) 

3 

13 

3 

13 

1 
6 
2 
1 

J» 

(.28, .68) 
(.04.. 22) 

4 

12 

4 

2 

6 
3 

5 

(.50, .86) 
(.14, .50) 

(.01, .12) 

12 

5 
11 

5 
11 

5 

11 

7 
2 

5 
3 

6 
4 
7 

6 

(.30, .74) 
(.06, .33) 
(.003, .05) 

6 
10 

6 
10 

6 
10 

3 

5 
4 
6 

5 
7 

7 

(.50, .88) 
(.16, .56) 
(.02, .18) 
(.001,  .02) 

7 
9 
7 
9 
7 
9 
7 
9 

3 

5 
5 

6 
6 

7 

(.31,-75) 

8 

4 

(.07,-36) 

8 

5 

8 

(.005, .07) 

8 

6 

(.0001, .003) 

8 

7 
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TABLE  k 
Results  of  First  Stage  Tests 


Group  Number 

(-r) 

Number  of  First 
Stage  Tests  in 
Group  (0^) 

Size 
Perf 

Used 
orm  Te 

to 
ssts 

Number  of 
Rejections 

1 

1 

c50 

0 

2 

3 

»23 

2 

3 

1 

,50 

1 

k 

3 

028 

0 

:'    5 

1 

o50 

0 

6 

^ 

o30 

1 

7 

8 

o50 

3 

8 

0 

.31 

0 

that  a  rather  large  number  of  near  optimal  vectors  (a) 
exist o   In  the  case  study  just  presented,  an  extended  in- 
vestigation revealed  that  there  are  at  least  2k   near  optimal 
vectors  for  a  test  of  size  o05.   (Indeed  there  may  be  more 
than  2k,    since  the  investigation  did  not  enumerate  all 
possible  vectors  cu  )   Table  5  presents  a  resume  of  the  near 
optimal  vectors  founds   It  is  interesting  that  at  least  20 
first  stage  size  vectors  yield  powers  which  differ  by  less 
than  o00^o   Additionally ,  the  investigation  disclosed  at 
least  26  more  vectors  which  produce  powers  greater  than  o97^ 
and  may  be  considered  "near=optimal"0 
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TABLE  5 
A  List  of  Near  Optimal  First  Stage  Size  Vectors 

Note:   a.  =  .50,  a2=e23,  and  a.o=o31  in  all  vectors.   Only 

the  subvector  (aQ,a.  ,a_.,a  ,,a0 )  is  tabulated. 

j     **     5   o  ( 


Subvect 

or 

TT* 

Subvect 

or 

TT« 

(.50 

,.28, 

.14, 

.30, 

.16) 

.98435 

(.10, 

.28, 

.01, 

.30, 

.16) 

.98138 

(.50 

,.28, 

.50, 

.30, 

.16) 

.98419 

;.io, 

.28, 

.01, 

.30, 

.50) 

.98054 

(.50 

,.28, 

.50, 

.30, 

.50) 

.98401 

:.5o, 

.72, 

.14, 

.30, 

.16) 

.98039 

(.50. 

,.28, 

.1*. 

.30, 

.50) 

.98391   l 

;.5o, 

.72, 

.50, 

.30, 

.16) 

.98035 

(.10 

,.28, 

.50, 

.30, 

.16) 

.98382   i 

;.9o, 

.28, 

.86, 

.30, 

.16) 

.98015 

(.10. 

,.28, 

.14, 

.30, 

.16) 

.93360   1 

;.5o, 

.72, 

.50, 

.30, 

.50) 

.97994 

(.10. 

,.28, 

.50, 

.30, 

.50) 

.98344 

:.5o. 

,72, 

.14, 

.30, 

.50) 

.97971 

(.10. 

,.28, 

.14, 

.30, 

.50) 

.98288   | 

:.9o, 

.72, 

.50, 

.30, 

.16) 

.97829 

(.90, 

,.28, 

.50, 

.30, 

.16) 

.98234   | 

:.5o, 

.04, 

.14, 

.30, 

.16) 

.97821 

(.50, 

,.28, 

.86, 

.30, 

.16) 

.98232   | 

;.5o, 

.04, 

.50, 

.30, 

.16) 

.97782 

(.90, 

.28, 

.14, 

.30, 

.16) 

.98220   I 

;.9o, 

.72, 

.14, 

.30, 

.16) 

.97768 

(.90, 

,.28, 

.50, 

.30, 

.50) 

.98205   ( 

;.5o, 

.28, 

.50, 

.70, 

.16) 

.97750 

(.50, 

.28, 

.86, 

.30, 

.50) 

.98190   ( 

.10, 

.04, 

.50, 

.30, 

.16) 

.97745 

(.90, 

.28, 

.14, 

.30, 

.50) 

.98174   ( 

.50, 

.72, 

.86, 

.30, 

.50) 

.97743 

(.50, 

.28, 

.01, 

.30, 

.16) 

.98169   ( 

.50, 

.28, 

.50, 

.06, 

.16) 

.97741 

(.50, 

.28, 

.01, 

.30, 

.50) 

.98161   ( 

.50, 

.28, 

.14, 

.06, 

.16) 

.97738 
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If  we  consider  an  experidment  somewhat  smaller  than 
in  the  preceeding  case  study,  it  is  possible  to  list  all 
possible  first  stage  size  vectors  (easily)  and  compute  the 
resulting  powers  n*0  Let  us  consider  the  case  n=l3»  m=8, 
and  suppose  the  total  responses  (1  '  s)  were  such  that 

0=0, "r=  1,2,3,4,8,  0,-1,  0.  =  ^,  and  0  =8.   In  this 

do  7 

instance  the  computer  code  of  Appendix  2  produced  values  of 
n»  for  all  possible  first  stage  size  vectors  in  less  than 
1  minute  on  the  IBM  3&0  computer.   The  optimal  vector  oT 
yielded  a  power  of  09^9«   For  small  experiments,  or  when 
less  than  5  of   the  0*s  are  non  zero,  this  complete  enumer- 
ation technique  is  recommended.   For  other  cases,  the  number 
of  possible  vectors  (a's)  is  generally  rather  large  and 
computation  time  grows  rapidly .   In  the  case  study  presented, 
a  complete  enumeration  would  have  required  more  than  4  hours 
of  computer  time,  which  is  prohibitive  for  most  users, 
especially  in  view  of  the  small  improvement  in  tt»  which  may 
result. 

COMMENTS  ON  POWER 

In  formulating  the  two  stage  test,  we  have  weakened 
some  of  the  assumptions  required  for  the  earlier  techniques 
considered  in  Chapter  le   We  would  therefore  expect  some 
reduction  in  power  compared  to  the  tests  which  take  advantage 
the  stronger  assumptions .   In  order  to  give  some  clarifi- 
cation of  this  point,  a  numerical  example  is  now  given. 
Suppose  the  experiment  discussed  in  the  case  study  had  been 
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analyzed  under  the  additional  assumption  that  all  individuals 

perform  equally.   Under  that  assumntion,  the  total  number  of 

21 
true  detections   2   j  =  97  and  the  total  number  of  false 

21       1=1   i 
alarms   £  1.  -  j,  =  98  would  have  been  considered  as  obser- 

1=1  A 

vations  on  the  random  variables  J*  and  K*  where  J'  =  b(l68,d) 

and  K'  =  b(l68,f).   Typically,  the  test  of  H  :  d=f  against 

1  _  f 
H-,  :  d-f=- : would  be  made.   Using  one  of  the  usual 

L2f]   +1 
asymtotic  techniques2  with  size  .05  results  in  a  power  of 

.999  at  the  point  (d=  .75,  f =  .5)  a  typical  point  on  the 

curve  used  in  the  alternate  hypothesis  for  the  case  study. 

Comparing  .999  with  .98^,  the  optimal  power  for  the  two 

stage  test  in  the  case  study,  we  see  that  relaxation  of  the 

assumptions  d.  =d  Vi,  and  f .  =  f  Vi  has  "cost"  .015  in 

power.   wfhile  it  is  for  the  individual  experimenter  to 

decide  on  the  merits  of  this  "trade  off",  it  seems  that  in 

general,  the  loss  of  power  is  slight  in  light  of  the  much 

broader  applicability  of  the  two  stage  test.   Additionally, 

at  the  point  (d=  .9,  f =  *7 '5)  which  is  also  on  the  curve 

d-f  =  —   j the  asymtotic  technique  yields  power  of  .980 

[2f]-1-j-l 
whereas  the  two  stage  test  continues  to  have  power  of  .98^. 

At  this  more  extreme  point,  the  two  stage  test  is  better 
than  the  asymtotic  test  under  the  stronger  assumptions. 

It  may  also  be  important  to  set  a  lower  bound  on  the 
power  of  a  two  stage  test  for  a  given  experiment  before  it 
is  run.   The  poorest  power  would  emerge  if  all  results  fell 
in  group  1,  that  is,  if  L1  was  1  or  m  -  1  for  all  i.   How- 
ever, the  result  0.  = n  is  so  unlikely  that  it  is  felt  the 
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hypothetical  result  0  =  n  or  even  0  =  n  can  be  used  to  give 
a  practical  lower  bound  for  power  which  is  still  rather 
conservative  (perhaps  even  pessimistic). 

In  the  case  study,  if  0?  was  21,  the  test  H  :  p  =  a 
against  H  :  p  =  n  would  be  used  to  determine  the  lower 
bound  on  rr*  ,  where  (oc2,tt  )  is  the  size  -  power  pair  which 
gives  the  best  power  at  size  „05»   Completing  this  procedure 
for  the  case  example  yields  .91  as  practical  lower  bound  on 
the  power  of  the  two  stage  test.   Of  course,  the  actual 
results  were  much  more  "informative"  than  0  =  n,  and  the 
actual  power  achieved  was  0^Q^^ 
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CHAPTER  V 
A  RAPID  APPROXIMATE  PROCEDURE 

If  we  strengthen  assumption  1  (pc  11)  by  adding  the 

statement  "m  is  sufficiently  large  to  assure  that  the  arcsine 

6  7 
transformation  ' '  provides  reasonably  'normal'  random 

variates  when  applied  to  the  experimental  data",  (leaving 
all  other  assumptions  as  stated),  we  may  then  use  an  approx- 
imate procedure  to  test  the  hypothesis 

H  :  d1  =  f1  Vi   against  E^s  &±  ^  f  V  i. 

Only  a  brief  derivation  will  be  given  and  we  make  no 
comments  about  the  properties  of  the  procedure  except  to 
remark  that  it  is  easy  and  rapid  in  application  as  compared 
to  the  two  stage  test,  and  in  all  test  cases  thus  far 
examined,  has  provided  the  same  results  (in  terms  of  re- 
jection of  H0)  as  the  two  stage  test. 

DERIVATION  OF  THE  PROCEDURE 

tie   shall  use  the  terminology  of  Chapter  2.   Under  the 
strengthened  version  of  assumption  1  noted  above,  we  have 
(approximately ) 


arcsin  -/J  /m  =  N(arcsin  -/d.  ,  1/^m)  \/i,  and 
arc  sin  -/  K.  /m  =  N  (arcsin  ^ff      ,  1/^m)  Vi. 
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From  the  well  known  properties  of  the  normal  distribution 

2m  arcs  in  ij  J.  /in  =  N(   2m  arcsin-/d   ,  1/2)  V  i,  and 
2m  arcsin  -/  K  /m  =     N(   2m  arcsin -J"?^  1/2)  yi. 

Further,  for  each  i,  i=l,2,oa.,n 

2m (arcsin  -7  J.  /m  -  arcsin  ^  K.  /m)  =  N(  2m  [arcsin-fd.  -  arcsin -(f.],l). 

Now  under  H  :  f „  =  d  V  i ,  we  have 
0   i   i 


2m 


(arcsin  y  J.  /m  -  arcsin -/  K  /m  )  =   N(0,1)  Vi. 


Denoting  2m  (arcsin  -j  J  /m,  -  arcsin -J  K. /m)  by  Z.  , 


we  have 

(Z.  )   -'L^    5  i=l,2,„.,,n,  so  that 


n      2   do,  2 


2  (Z.  f     £   *$  . 

i=l 


i        n 


In 


order  to  test  H  :  d  =  f   Vi  against  H.  :  d.  ^  f   Vi   at 

Oil  l    l    i 


n       2   o,2 
size  a,  we  may  reject  H  if  £   (Z  )   >  -{„  «  , 

0    j__-]_     i      n»  L~'J 
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APPENDIX  A 

A  FORTRAN  CODE  FOR  COMPUTING  OPERATING  CHARACTERISTIC 
CURVES  FOR  FISHER'S  EXACT  TEST  OF  PERCENTAGES 

The  code  which  follows  evaluates  equation  19  of 
Chapter  2  appropriately  to  form  the  complementary  cumulative 


distribution  functions  of  J 


1  for  each  1.  ,  1,  =1,2,..,  ,2m 
i  l   i 


under  both  the  null  and  alternate  hypothesis.   The  input 

parameters  PD,  PF,  and  M  correspond  respectively,  to  d. , 

f  ,  and  m  in  equation  19.   The  values  of  PD  and  PF  used  as 
i 

input  to  the  code  can  be  any  point  on  the  alternate  hypothesis 
curve  d.  -  f .  =  (1  -  f  )/ j  [f .  (D  -  1 )]"  +  1  j .   (The  program 
is  useful,  of  course,  for  any  investigation  of  Fisher's 
Test  whether  or  not  related  to  the  specific  use  intended 
here.  ) 

A  subroutine  format  is  used  to  allow  maximum  flexi- 
bility in  the  application  of  the  code.   Depending  on  the 
computer  used,  this  subroutine  should  perform  for  values  of 
m  up  to  at  least  40.   The  output  includes  a  full  definition 
of  parameters  and  variables  and  is  self-explanatory. 


59 


SUBROUTINE  HYPER < M.PF , PD> 
P=PD*(1-PF)/PF/(1-PDI 
0=1. O/P 
WRITE(6,4I 

4  F0RMAT(1H1,4X, 'DEFINITION  OF  SYMBOLS :•,//, 10X,  •  M  IS' 
i«  THE  NUMBER  OF  DETECTION  ATTEMPTS  MADE  UNDER  EACH  • 
2»TEST  CONDITION. '//.lOXt'L  IS  THE  TOTAL  NUMBER  OF  • 
3UNDICATED  DE TECT IONS. • // ,10X . • J  IS  THE  NUMBER  OF  • 
^•CORRECT  DETECTIONS. V/tlOX, »S0  IS  THE  PROBABILITY  ■ 
5»0F  OUTCOMES  STRICTLY  GREATER  THAN  J  UNDER  HO.1//, 
610X,«S1  IS  THE  PROBABILITY  OF  OUTCOMES  STRICTLY  • 
7»GREATER  THAN  J  UNDER  Hl.V/l 

WRITE(6,8)M,PF,PD,PF 

8  F0RMAT(5X, 'DISTRIBUTION  FOR  M=« I  2 , • . • // , 5X, • HO:  • 
IMPROBABILITY  OF  FALSE  ALARM  =  PROBABILITY  OF  TRUE  ■ 
2»DETECTI0N  =  »F6.4  ,».»// ,5X, • HI :  PROBABILITY  OF  TRUE* 
31  DETECTION  = »F6. 4  ,».«// ,9X ,« PROBABIL ITY  OF  FALSE  • 
4*ALARM  =»F6.4, •.«//) 

WRITE(6,5) 

5  F0RMAT(5X, • L »8 X, • J »9X , • SO • 1 3X , • SI ■ 12X, • L • 8X, • J • / ) 
DO    50       L=1,M 

WRITEI6.7) 
7    FORMATUX,//) 
SC=1.0 
Sl=1.0 
LL=L*1 

DO  50  JJ=1 fLL 
J=JJ-1 
XJF=1.0 
IFCJ-1111,11,9 

9  DO  10  K=1,J 
XK=K 
XJF=XJF*XK 

10  CONTINUE 

11  XLJF=1.0 
LJ=L-J 
IF(LJ)14, 14,12 

12  DO  13  K=1,LJ 
XK  =  K 
XLJF=XLJF*XK 

13  CONTINUE 

14  XMJF=1.0 
MJ=M-J 
IF(MJ)17, 17,15 

15  DO  16  K=1,MJ 
XK=K 
XMJF=XMJF*XK 

16  CONTINUE 

17  XMJLF=1.0 
MJL=M*J-L 
IF(MJL-1)2C,20,18 

18  DO  19  K=1,MJL 
XK  =  K 
XMJLF=XMJLF*XK 

19  CONTINUE 

20  W=0.0 
U=0.0 

DO  40  KKA=1,LL 
KA=KKA-1 
XAF=1.0 
IF(KA-1I23,23,21 

21  DO  22  K=1,KA 
XK=K 
XAF=XAF*XK 

22  CONTINUE 

23  MA=M-KA 
XMAF=1.0 
IF<MA-1)26,26,24 

24  DO  25  K=1,MA 
XK  =  K 
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XMAF=XMAF*XK 

25  CONTINUE 

26  LA=L-KA 
XLAF*1.0 
IF(LA-1)29,29,27 

27  00  28  K=1,LA 
XK  =  K 
XLAF=XLAF*XK 

28  CONTINUE 

29  MAL  =  M«-KA-L 
XMALF=l.O 
PQ=1.0 
IF(MAL-1)32,32,30 

30  DO  31  K=1,MAL 
XK  =K 
XMALF=XMALF*XK 

31  CONTINUE 

32  IF(KA-J)33,35,34 

33  PQ=Q**(J-KA) 
GO  TO  35 

34  PQ=P**(KA-JI 

35  UA=(XJF/XAF)*(XIJF/XLAFI*(XMJF/XMAF)*<XMJLF/XMALF> 
PQ=PQ*UA 

U=U*UA 
W=W*PQ 
40  CONTINUE 
R=1.0/W 
S=1.0/U 
ML=2*M-L 
SO=SO-S 
IF(S0I60,61,61 

60  S0=0.0 

61  S1=S1-R 
IF(S1)62,63,63 

62  S1=0.0 

63  WRITE(6,6)L,J,   SO,   Sl.Ml.MJL 

6  F0RMAT(4X,I2,7X,I2,7X,2(F10.8,5X),I2,7X,I2f//) 
50  CONTINUE 
END 
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APPENDIX  B 

A  FORTRAN  CODE  FOR  DETERMINING  THE  ACCEPTANCE  REGION 
FOR  THE  SECOND  STAGE  TEST  IN  THE  TWO  STAGE  PROCEDURE 

In  general,  this  program  can  be  used  to  generate  the 
CDF's  for  two  joint  distributions  of  up  to  12  independent 
binomial  distributions  with  first  parameters  given  by 
(KPHI(l),  KPHI(2),eoo,KPHI(12))  in  both  cases  and  second 
parameters  given  by  (ALPHA(1 ) ,ALPHA{2 ) , . 0 , .ALPHA (12 ) )  for 
the  first  joint  distribution  and  by  (XPI(l),  XPI(2 ),..., 
XPI(12))  for  the  second,, 

The  program  input  parameters  which  relate  to  the 
prototype  case  variables  are: 


KPHI(I)  -  0. 


Number  of  first  stage  tests  in  the  i 
group o 


yh 


ALPHA(I)~a.   Size  of  first  stage  tests  in  the  i 


th 


XPI(I) 


M 


th 


group. 

Power  of  first  stage  tests  in  the  i' 
group  e 

Number  of  groups  which  contain  at  least 
one  test,   (If  desired,  all  groups,  in- 
cluding those  for  which  0.  =  0  can  be  read 
in  to  the  program  but  this  practice 
increases  computing  time.   If  this  prac- 
tice is  followed,  then  M  corresponds 
to  m0  ) 
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Other  program  input  parameters  are: 

START   The  largest  size  for  which  a  printed  value  of 
a'  and  n'  is  desired. 

SIZE   The  smallest  size  for  which  a  printed  value  of 
a'  and  it*  4s  desired, 

KP     The  Kind  of  print  parameter. 

KP  =  0  provides  a  complete  printout  of  sizes  and 

corresponding  powers  for  all  sizes  between  SIZE 

and  START  plus  a  summarized  description  of  the 

acceptance  region  for  a  test  of  size  SIZE. 

KP=-1  provides  only  the  print  out  of  sizes  and 

corresponding  powers  for  all  sizes  between  SIZE 

and  START. 

KP  =  1  provides  only  the  summarized  description 

of  the  acceptance  region  for  a  test  of  size  SIZE 

plus  a  statement  of  the  power  of  that  test. 

NOTE:   Regardless  of  the  KP  parameter  specified, 

the  output  also  always  includes  a  recap  of  the 

0.'s,  oc  •  s,  and  rr  *s  for  verification  purposes. 
1     i        i 

In  addition  to  reading  in  the  above  parameters,  it  is 
also  necessary  to  insure  that  the  dimension  of  KSUM,  XLAM, 

and  KX  is  greater  than  or  equal  to  v,  which  can  be  computed 

m 
by  the  formula  v=  jr   (0.  +1).   With  KSUM,  XLAM,  and  KX  so 

1=1   x 

dimensioned,  the  program  will  operate  on  any  experiment  for 

which  the  number  of  subjects  (n)  is  less  than  or  equal  to 
30  and  for  which  k,  the  number  of  non-zero  0.'s  is  less 
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than  or  equal  to  12 0   The  program  is  readily  extended  to 
larger  problems  but  experience  indicates  that  such  cases 
cause  excessive  computing  time  and/or  exceed  computer 
storage  capacity. 
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INTEGER*2  KSUM.KX 

DIMENSION  KPHI ( 12 > , A( 21 , 21 1 , KT( 31 ) , ALPHA ( 12) , 
1ABAR<12).XPI(12).pIbAR<12),XL<12>,XLBAR<12),LL< 12  It 
2 ML < 12  I, PA( 2 1,21, 12) , NY( 31 ) ,KFY ( 31 ) , 
3XLAM<5760) ,KX( 5760) ,KSUP< 5760 > 

READ< 5.2) M, START. KP.SIZE 

2  F0RMAT(l8fF8.0?I8,F8\0) 

RE AD (5.1) (KPHI U) ,ALPHAU),XPI( I),I=1,M) 
1  F0RMAT(I8,2F8.0) 
C 

C      INITIALIZE   ARRAYS  AS  NECESSARY 
C 

DO  10  1=1,31 

NYU  1=0 

KFY(I)=0 

10  CONTINUE 

DO  11  1=1,12 
LL(I)=0 

11  CONTINUE 

DO  16  1=1,31 
KT(I)=0 

16  CONTINUE 
KYT  =  1 

C 

C      COMPUTE  AUXILIARY  PARAMETERS  AND  ONE  PLUS  THE 

C      LARGEST  POSSIBLE  NUMBER  OF  SUCCESSES  (KYT) 

C 

DO  15  1=1, M 

ABAR(I)=1.0-ALPHA( I) 

PIBAR(I)=1.0-XPI( I ) 

XL( I)=ALPHA(I)/XPI(l) 

XLBAR(I)=ABAR(I)/PIBAR(I) 

KYT=KYT+KPHI (I ) 
15  CONTINUE 
C 

C      SET  UNNEEDED  PARAMETRS  TO  THEIR  IDLE  VALUES 
C 

IF(M-12)17,19,19 

17  MMsM+1 

DO  18  I=MM,12 
KPHI(I)=0 

18  CONTINUE 
C 

C      COMPUTE  THE  NUMBER  OF  DIFFERENT  VECTORS  POSSIBLE 
C 

N=l 

19  DO  3  1=1,12 
ML(I)=KPHIU)  +  1 
N=N*ML(I ) 

3  CONTINUE 
C 

C      SET  PARAMETERS  NEEDED  FOP  ENSUING  DO  LOOPS 
C 

M1=ML(1) 

M2=ML(2) 

M3=ML(3) 

M4=ML(4) 

M5=ML(5) 

M6=ML(6) 

M7=ML(7) 

M8=ML(8) 

M9=ML(9) 

M10=ML(10) 

M11=ML(11) 

M12=ML(12) 

WRITE<6,200) 
200  F0RMAT(1H1,//) 

NPLUS=N*1 

JT=ML(1) 

DO  700  I=2,M 

IF(JT-ML(I )) 701, 700, 700 
701  JT=ML(I) 
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700  CONTINUE 
C 

C      COMPUTE  COEFFICIENTS  FOR  BINOMIAL  PROBABILITY 
C      EXPRESSIONS 
C 

DO  20  1=1,  JT 

XI  =  I 

or  20  J  =  l ,T 

XJ  =  J 

XJBAR=XI-XJ+1.( 

A( I  ,  J)=GAMMA(XI  1/ (GAMMA(XJ)*GAMMA(XJBAR)  ) 
C 

C      MULTIPLY  COEFFICIENTS  BY  APPROPRIATE  POWERS  OF  "P" 
C      AND  "0".  PA(I,J,K)  IS  THE  PROBABILITY  OF  J  SUCCESSES 
C      GIVEN  I  TRIALS  WITH  SUCCESS  PROBABILITY  ("P")  EQUAL 
C      TO  XPHKI. 
C 

IF( J-l)5,5,6 

5  IF( I-J)7,7,8 

7  DO  21  K=1,m 

PA(  I  ,J,K)=A(  I ,  J) 

21  CONTINUE 
GO  TO  20 

8  DO  22  KK=1 f M 

IF(  I-ML(KK)  ) 301  ,301,22 

301  PA< I  ,J,KK)=A( I  , J)*(PIPAR<KK)**<  I-JII 

22  CONTINUE 
GO  TO  20 

6  IFU-J)9,9,4 

9  DO  23  KKK=1,M 

IF(  I-ML(KKK)  )  3f,2,302,23 

302  PA(  I  ,  J,KKK)=A(  I  ,J)*(XPI <KKK>**< J-l)  ) 

23  CONTINUE 
GO  TO  20 

4  DO  24  KKKK  =  1  ,M 

IF( I -ML (KKKK 1)303, 303, 2^ 

303  PA( I , J,KKKK>=A( I , JI*(XPI(KKKK)**< J-l) )* 
1(PIBAR(KKKK)**( I-J) ) 

24  CONTINUE 
20    CONTINUE 

C 

C      COMPUTE  LIKELIHOOD  RATIOS  FOR  ALL  VECTORS  (XLAM(L)) 


C 


L=0 

DO  90  J1=1,M1 

K1=J1-1 

KB1=M1-J1 

IF (Kl) 30, 30,31 

30  B=XLBAR(1)**KB1 
GO  TO  34 

31  IFfKBl >32,32,33 

32  B=XL( 1)**K1 
GO  TO  34 

33  B=(XLBAR(1 >**KB1)*(XL(1 )**K1) 

34  DO  90  J2=l,*2 
K2=J2-1 
KB2=M2-J2 
IF(K2)35,35,36 

35  IF(KB2)91.91,37 

37  C=B*XLBAR(2)**K82 
GO  TO  40 

36  IF(KB2)33,38,39 

38  C=B*XL(2)**K2 
GO  TO  40 

3  9  C=B*( XLBAR(2)**KB2I*(XL(2)**K2> 

GO  TO  40 
91  C  =  B 

40  DO  90  J3  =  l  ,M3 
K3=J3-1 
KB3=M3-J3 
IF(K3)41,41,42 

41  IF(KB3>92,92,43 
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43  D*C*XLBAR(3I**KB3 
GO  TO  46 

42  IF(KB3)44.44.45 

44  D=C*XL(3»**K3 
GO  TO  46 

45  D=C*(XLBARm**KB3)*(XL<3)**K3> 
GO  TO  46 

92  0=C 

46  DO  90  J4=1,M4 
K4=J4-1 
KB4=M4-J4 
IFCK4I47.47.48 

47  IF(KB4>93.93,49 

49  E=D*XLBAR(4)**KB4 
GO    TO    52 

48  IF(KB4)50. 50.51 

50  E=0*XL(4)**K4 
GO    TO    52 

51  E=D*(XLBAR(4|**KB4>*(XL(4)**K4> 
GO    TO    52 

93  E=D 

52  DO    <50    J5  =  1,M5 
K5=J5-1 
KB5=M5-J5 
IFCK5I53.53.54 

53  IFCKB5)94,94,55 

55  BB=E*XLBAR(5)**KB5 
GO  TO  58 

54  IFCKB5)56,56,57 

56  BB=E*XLC5>**K5 
GO  TO  58 

57  BB=E*CXLBARC51**KB5)*CXLC5J**K5) 
GO  TO  58 

94  BB  =  E 

58  DO  90  J6=1,M6 
K6*J6-1 
KB6=M6-J6 
IFCK6)59,59,60 

59  IFCKB6)95,95,61 

61  CC=BB*XLBARC6)**KB6 
GO  TO  64 

60  IF(KB6)62, 62,63 

62  COBB*XLC6»**K6 
GO  TO  64 

63  CC  =  BB*CXLBARC6)**<KB6)*CXLC6>**K6) 
GO  TO  64 

95  CC=BB 

64  DO  90  J7=lfM7 
K7=J7-1 
KB7=M7-J7 
IFCK7)65,65,66 

65  IFCKB7)96. 96,67 

67  DD=CC*XLBARC7I**KB7 
GO  TO  70 

66  IF(KB7)68, 68.69 

68  DD=CC*XLC7I**K7 
GO  TO  70 

69  DD=CC*CXLBARC7)**KB7)*CXLC7)**K7) 
GO  TO  70 

96  DD=CC 

70  DO  90  J8=1,M8 
K8=J8-1 
KB8*M8-J8 
IFCK8)71,71,72 

71  IF(KB8»97.97.73 

73  EE=DD*XLBAPC8>**KB8 
GO  TO  76 

72  IFCKB8J74.74.75 

74  EE=DD*XL(8»**K8 
GO  TO  76 

75  EE»DD*CXLBARC8I**KB8)*CXIC8)**K8) 
GO  TO  76 
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97  EE=DD 

76  DO  90  J9=l ,M9 
K9=J9-1 
KB9=M9-J9 
IF(K9)77,77, 78 

77  IF(KB9»98, 98,79 

79  FF=EE*XLBAP(9>**KB9 
GO  TO  82 

78  IF(KB9)80,P0,81 

80  FF=EE*XL(91**K9 
GO  TO  82 

81  FF  =  EE*<XLBAR(9)**KB9)*(XL(9H<*K9) 
GO  TO  82 

98  FF=EE 

82  00  90  J10=1,M10 
K10=J10-1 

KB10  =  M10-JU 
IF(K10)83,8?,84 

83  IF( KB  101 99,9  9,8  5 

85  GG=FF*XLBAR( 10 1 **KB10 
GO  TO  8R 

84  IF(KB10)86,86,87 

86  GG=FF*XL(101**K10 
GO  TO  88 

87  GG=FF*( XLBAR< 10 1 **K810> *( XL ( 10 1 **K10 » 
GO  TO  88 

99  GG=FF 

88  00  90  J11=1,M11 
K11=J11-1 
KB11=M11-J11 
IF(K11)240,240,241 

240  IF(KB111100, 100,242 

242  R=GG*XLBAR(11)**K811 
GO  TO  245 

241  IF(KB111243,243,244 

243  R=GG*XL( 11)**K11 
GO  TO  245 

244  R=GG*(XLBAP( 11 1 **KB1 1 1  * ( XL( 1 1 )**K1 1 1 
GO  TO  245 

100  R=GG 

245  DO  90  J12=  1,M12 
K12=J12-1 
KB12=M12-J12 
IF(K12)246,246,247 

246  IF(KB12)101, 101,248 

248  RR=R*XLBAR(12!**KB12 
GO  TO  251 

247  IF(KB12)249, 249,250 

249  RR=R*XL(12)**K12 
GO  TO  251 

2  50  RR=R*(XLBAR< 12)**KB12)*(XL(12)**K12) 
GO  TO  251 

101  RR=R 
C 

C      ENCODE  THE  VECTOR  (K1,K2,  ••.,K12)  INTO  THE  NUMBER  L 
C 

251  L=L+1 
C 

C      COMPUTE  THE  SUM  OF  COMPONENENTS  IN  THE   VECTOR 
C      (K1,K2,... ,K12> 
C 

KY  =  J12  +  K11  +  K10«-K9+K8  +  K7  +  K6*K5>K4+K3  +  K2+K1 
C 

C      COUNT  THE  NUMBER  OF  VECTORS  WHOSE  SUM  IS  KY 
C 

NY<KY)=NY(KY)*1 
C 

C  ENTER    THE    LIKELIHOOD    RATIO    AND    THE    ENCODED    VALUE 

C  OF    ITS    ARGUMENT    INTO    THEIR    RESPECTIVE    ARRAYS 

C 

XLAM(L1=RR 

KX(L1=L 
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90  CONTINUE 
C 

C      SORT  THE  LIKELIHOOD  RATIOS  INTO  AN  OROERED  ARRAY 
C 

NMIN=N-1 
DO  380  I=l,NMIN 
11=1*1 

DO  381  J  =  II  ,N 

IF(XLAM(I)-XLAM(J) I  382 1 38 1,381 
382  T=XLAM(J) 
IT=KX( J) 
XLAM( J>=XLAM(I ) 
KX( J)=KX(I) 
XLAMU  >  =  T 
KX(I)=IT 
381  CONTINUE 
380  CONTINUE 
C 

C      PRINT  GENERAL  HEADINGS 
C 

F  =  1.0 
0  =  1.0 

WRITE<6.209MKPHI  (  I)  ,1  =  1. Ml 
209  F0RMATUH1, 'NUMBER  CF  TRIALS  AND  ASSOCIATED  SUCCESS' 
i»  PROBABILITIES  UNDER  THE  NULL  AND  ALTERNATE  HYPOTH' 
2'ESES'///,3X,'NUMBER'/,1X,'0F  TRI ALS: • 3X, 12 ( I  2, 4X ) ) 
IF(KP>410.410,411 

411  WRITE(6,416) 

416  FORMATUX,//! 
GO    TO    417 

410  WRITEC6.405) 

405  F0RMAT<lH+,90X,'NnTE:  THE  BELOW  L ISTED' / , 89X , • CUMUL ' 
l'ATIVE  PROBABILITIES  ARE* / ,89X, • DEF INED  AS  THE  PROS' 
2'ABILITY') 

417  WRITE(6,208MALPHA(I),I=1,M» 

208  FORMAT! 5X, • ALPHA : • 2X, 12 (F4.  2 , 2X ) ) 
IF(KPI412,412,413 

413  WRITE<6,416> 
GO  TO  418 

412  WRITE(6,406» 

406  F0RMAT(lH«-,88X,'THAT  THE  VECTOR  TO  THE  LEFT'/,89X, 
l'OR  ANY  BELOW  IT  (THAT  IS , • / ,89X, • W ITH  SMALLER  LIKE* 
2'LIHOOD) •) 

418  WRITE(6.207MXPim,I=l,M! 
207  FORMAT (8X, 'PI : • 2X ,12( F4. 2 ,2X I ) 

IF(KP)414,414,261 
261  WRITE<6,416) 
GO  TO  419 

414  WRITE(6.407) 

407  F0RMAT(1H*,88X, 'OBTAINS'/) 
WRITE(6.206) 

206  F0RMAT(2X,«SUM  OF '22X  ,• NUMBER  OF  SUCCESSES'40X, 'CUM' 
1'ULATIVE'5X,'CUMULATIVE'/,1X,'SUCCESSES'79X, 'PROBAB' 
2ULITY»4X,'PR0BABILITY'/,89X, 'UNDER  H0'7X, 'UNDER  HI' 
3//) 
C 

C      RECOVER  THE  VECTORS  ( K1.K2,. . . ,K12 )  AS  (LL  ( 1 )  ,LL ( 2  I , 
C      ...,LL(12))  STARTING  WITH  THE  ONE  WITH  GREATEST 
C      LIKELIHOOD  RATIO,  THUS  FORMIMG  THE  ACCEPTANCE 
C      REGION 
C 

419  DO  170  L=1.N 
KU=KX(L)  -1 
MB=13-M 

DO  160  I=MB,12 

K=13-I 

LL(K)=MOD(KU,ML(KI) 

KU=KU/ML(K) 
160  CONTINUE 
C 

C      COMPUTE  THE  PROBABILITY  MASS  FOR  THE   VECTOR 
C      (LL(1),LL<2) t««.LL(12)l  UNDER  HO  AND  HI 
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U  =  1.0 

DO  165  1=1, M 
420  KL=LL( I  1  +  1 

U=U*PA(ML(I I  ,KL,I  I 
165  CONTINUE 

W=XLAM(L)*U 
C 

C      DECREMENT  CUMULATIVE  DISTRIBUTION  FUNCTIONS 
C 

V=F 

F=F-W 

VV=Q 

0=0-U 

LY  =  l 
C 
C      RECOVER  SUM  OF  COMPONENTS  OF  VECTOR 


C 


DO  168  1=1.12 
LY=LY+LLU I 

168  CONTINUE 


C 

C      TAG  THE  VECTOR  WHOSE  ENCODED  VALUE  IS  L  WITH  THE 

C      VALUE  OF  THE  SU*  OF  ITS  COMPONENTS 

C 

KSUM(L>=LY 
C 

C      CCUNT  THE  NUMBER  OF  VECTORS  SO  FAR  ACCOUNTED  FOR 
C      WHOSE  SUM  IS  LY 
C 

KFY(LY)=KFY<LY>*1 

IF(KP)262,262,263 
C 

C      START  PRINTING  THE  DISTRIBUTIONS  WHEN  THE  SIZE 
C      VALUE  REACHES  START 
C 

262  IF(START-F>263, 264,264 
264  KZ=LY-1 

WRITE(6,210>KZ 
210  FQRMAT(4X,I4) 

WRITEC6.408)  <LL(I ) ,I  =  1,M> 

408  F0RMAT(1H+,14X, 12(12, 4X) ) 
WRITEC6, 4091V, VV 

409  FORMAT(lH+,88X,2<F7.5,8XI,//» 
C 

C      STOP  PRODUCING  ADDITIONAL  VECTORS  WHEN  THE  DESIRED 

C      SIZE  VALUE  IS  REACHED 

C 

263  IF(SIZE-FU70,1T0,415 

170  CONTINUE 
C 

C      IF  CALLED  FOR,  PRINT  SUMMARY 
C 

415  IFCKPJ267, 171,171 

171  WRITE(6,225)V 

225  FORMAT(5X, 'THE  FOLLOWING  IS  A  SUMMARIZED  STATEMENT  • 
l«OF  THE  REJECTION  AND  ACCEPTANCE  REGIONS  FOR  A  TEST* 
2«  OF  SIZE'F6.3V •• •//) 
C 

C      FOR  THOSE  VECTORS  WHOSE  SUM  OF  COMPONENTS  IS  I  AND 
C      WHICH  WERE  ALL  IN  THE  ACCEPTANCE  REGION  ASSIGN 
C      KTU  1  =  2  TO  SO  DENOTE 
C 

DO  180  I=1,KYT 
NY(  I  )=NY(  II-KFYU  ) 
IFCNYCI  ni81,181,182 
181  KT(I)=2 

GO  TO  180 
C 

C      FOR  THOSE  VECTORS  WHOSE  SUM  OF  ELEMENTS  IS  I  AND 
C      WHICH  WERE  ONLY  PARTIALLY  ACCOUNTED  FOR  IN  THE 
C      ACCEPTANCE  REGION  ASSIGN  KT(I)=1  TO  DENOTE  THAT 
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C      A  MORE  OETAILED  INVESTIGATION  IS  REQUIRED.  FOR  THOSE 
C      VECTORS  WHOSE  ELEMENTS  SUM  TO  I  AND  FOR  WHICH  NONE 
C      APPEAR  IN  THE  ACCEPTANCE  REGION  SET  KT(I)=0. 
C 

182  IF(KFYU  11183,183,184 

183  KT(II=0 
GO  TO  180 

184  KT( I)=l 
180  CONTINUE 

WRITE(6.23G)V 

230  F0RMAT(6X. 'ALL  VECTORS  WHOSE  COMPONENTS  SUM  TO  THE  • 
1 'FOLLOWING  NUMBERS  ARE  IN  THE  ACCEPTANCE  REGION  FOR* 
2«  A  TEST  OF  SI ZE ■ F6.3  ,•••// ) 

C 

C      PRINT  THOSE  VECTORS  OF  THE  ACCEPTANCE  REGION  WHICH 

C      CAN  BE  IDENTIFIED  SIMPLY  BY  THE  SUM  OF  COMPONENTS. 

C 

DO  190  1=1, KYT 

K  =  I-1 

IF(KT(I)-1)191,191,192 
192  WRITE(6.23l»K 

231  FORMAT(25X,I3,//) 
KJ=I«-1 

190  CONTINUE 
C 

C      INVESTIGATE  VECTORS  FCR  WHICH  KT(I)=1  AND  PRINT 

C      THOSE  WHICH  ARE  IN  THE  ACCEPTANCE  REGION. 

C 

191  IF(KYT*1-KJ)267,267,266 

266  DO  195  I=KJ,KYT 
IF<KTmil97,197,194 

194  KI=I 

195  CONTINUE 

197  IF(KI-KJ)120,196,196 

196  WRITE(6,232)V 

232  F0RMAT(6X. 'THE  ACCEPTANCE  REGION  FOR  A  TEST  OF  SIZE* 
1F6.3,1X,»ALS0  INCLUDES  THE  FOLLOWING  VECTORS: •//, 7X, 
2 • SUM  OF • / , 5 X , • CCMPONE NTS • 30 X , • V ECTOR • // ) 

DO  110  I=KJ,KI 
DO  110  J=1,L 
IF<KSUM(J|-I) 110,111,110 

111  KU=KX(J)-1 

DO  112  K*MB,12 
KK=13-K 

LL<KK)=MOD<KU,ML(KK») 
KU=KU/ML(KK) 

112  CONTINUE 
KZ=KSUM< J)-l 
WRITEI6.237IKZ 

237   F0RMAT(10X,I2) 

WRITEI6.233)     < LL( K| .K=l .M » 

233  F0RMAT(1H+,25X, 10(12, 4Xh 
WRITE(6,236) 

236    FORMATdX,/) 

110  CONTINUE 
C 

C      PRINT  THE  CONCLUDING  STATEMENTS 
C 

120  WRITE(6,234»V 

234  F0RMAT(6X.»ALL  CTHER  VECTORS  ARE  IN  THE  REJECTION  R» 
l'EGION  FOR  A  TEST  OF  SIZE»F6.3, • • •//! 

WRITE(6f235)VV.V 

235  F0RMAT(6X,'THIS  TEST  HAS  POWER  OF* F6. 3, IX, • AT  SIZE»F6. 

267  CONTINUE 
END 
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